One file, one path: from
package main
to shipping a concurrent, observable Go service that fronts your models and never falls over.Every example is drawn from what AI engineers actually build in Go — streaming proxies, tool dispatchers, rate limiters, worker pools, context-cancelled model calls. No
foo
/bar
filler.
Companion reads: 🐍 Python for AI Developers (the sibling to this guide), 🏗️ Building High-Quality AI Agents, 🏢 Enterprise-Ready AI Agents.
| You are… | Start at | Skip |
|---|---|---|
| New to Go | Part 1 → read straight through | Parts 12–13 on first pass |
| Coming from Python | Part 1 (the phrasebook), then Part 5 and Part 6 | — |
| Coming from Java/C# | Part 4, Part 5 — inheritance and exceptions are gone | Part 2 (skim) |
| Building AI services | Part 6, Part 7, Part 9 | — |
| Reviewing code | Part 14, Part 15 | everything else |
Convention: // ✅
= do this, // ❌
= don't. Snippets target Go 1.22+, with newer-version wins called out inline.
Go was designed for large teams maintaining network services over years. Every trade-off follows from that:
| Go chose | Instead of | Consequence for you |
|---|---|---|
| A tiny spec (25 keywords) | Rich features | You can read any Go file after a week |
| Compile to one static binary | Runtime + deps | |
FROM scratch images, 10 ms cold start |
||
| Explicit errors as values | Exceptions | Failure paths are visible in the code |
| Composition + interfaces | Inheritance | No class hierarchies to reverse-engineer |
| Goroutines + channels | Callbacks / async colouring | Blocking code that scales to 100k connections |
| One formatter, one toolchain | Ecosystem choice | Zero config debates; go test , go fmt , pprof are built in |
Go is boring on purpose. The payoff is that a service written by someone who left two years ago still compiles, still reads clearly, and still runs.
[your .go files] → [compiler: types, escape analysis, inlining] → [one native binary]
↑ includes the runtime
(scheduler + GC)
COPY binary /
.The cost: more ceremony up front, no REPL, and a smaller ML ecosystem.
| Dimension | Go | Python |
|---|---|---|
| Execution | Native binary + embedded runtime | Bytecode on the CPython VM |
| Typing | Static, enforced by the compiler | Dynamic; static only via mypy in CI |
| Parallelism | Real: goroutines across all cores | GIL-limited; processes or C extensions |
| Concurrency cost | ~2 KB per goroutine | ~KB per coroutine, ~MB per thread |
| p99 latency | Stable (GC s < 1 ms) | Noisier |
| Deploy artifact | 15–40 MB static binary | Interpreter + wheels + lockfile |
| Startup | ~5 ms | 100–500 ms (imports) |
| ML/AI libraries | Thin (inference clients, ONNX, tokenizers) | Everything |
| Best at | API gateways, streaming proxies, orchestrators, high-fan-out workers | Model training, data science, ML inference glue |
The production shape that wins — and the one in this repo's CLAUDE.md — is both: Go as the BFF that owns HTTP, auth, tenancy, streaming and fan-out; Python as the ML service it calls for heavy computation. Use Go where request volume and connection count live; use Python where the models live.
| Python | Go | Note |
|---|---|---|
x = 5 |
||
x := 5 |
||
:= declares + infers, inside functions only |
||
list[int] |
||
[]int |
||
| Slice — dynamic array | ||
dict[str, int] |
||
map[string]int |
||
| Iteration order is randomized | ||
tuple |
||
| struct, or multiple return values | No tuple type | |
None |
||
nil (pointers, slices, maps, interfaces, funcs, chans) |
||
| Value types have zero values instead | ||
Optional[T] |
||
*T , or (T, bool) , or (T, error) |
||
| Pointers are the "maybe" of Go | ||
raise ValueError(...) |
||
return fmt.Errorf("...: %w", err) |
||
| Errors are returned, not thrown | ||
try/except |
||
if err != nil { … } |
||
| Explicit at every call | ||
with open(...) as f: |
||
f, err := os.Open(...) ; defer f.Close() |
||
defer is the context manager |
||
@decorator |
||
| Higher-order function / middleware | Wrap the function or the handler | |
class A: def m(self) |
||
type A struct{} + func (a A) M() |
||
| Methods live outside the type | ||
Protocol (structural) |
||
interface |
||
Go interfaces are structural too — no implements |
||
async def / await |
||
just call it, in a go routine |
||
| No function colouring | ||
asyncio.gather |
||
errgroup.Group |
||
Bounded with SetLimit |
||
asyncio.Semaphore(8) |
||
buffered channel or SetLimit(8) |
||
f"{x:.2f}" |
||
fmt.Sprintf("%.2f", x) |
||
pytest |
||
go test ./... |
||
| Testing is in the stdlib | ||
venv + pyproject.toml |
||
go.mod |
||
| Modules, no activation |
package main
import (
"fmt"
"log/slog"
"net/http"
"os"
)
func main() {
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
mux := http.NewServeMux()
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "ok")
})
logger.Info("listening", "addr", ":8080")
if err := http.ListenAndServe(":8080", mux); err != nil {
logger.Error("server failed", "err", err)
os.Exit(1)
}
}
Three things a Python developer should notice: no framework, no decorators, and errors returned rather than raised. ("GET /healthz"
method-and-pattern routing is Go 1.22+.)
🎯 Actionable rules
- Choose Go for the request path and the fan-out; keep Python where the models are.
- Let the compiler carry the weight you spend mypy effort on in Python.
- Learn
error
,interface
,defer
, andcontext
— everything else is syntax.
var name string // "" — declared variables are ALWAYS initialized
var count int // 0
var ratio float64 // 0
var ok bool // false
var tools []string // nil (usable: len 0, append works)
var index map[string]int // nil (readable, but WRITING panics)
var client *http.Client // nil
model := "claude-opus-5" // := infers the type; functions only
timeout, retries := 30, 3 // multiple assignment
_, err := doThing() // _ discards a value you must accept
Zero values are Go's answer to None. There is no uninitialized memory, so a struct is useful the moment it exists. Design your types so the zero value works (
sync.Mutex
, bytes.Buffer
, and http.Client
all do).⚠️ var m map[string]int
is nil: reads return the zero value, writes panic. Always m := make(map[string]int)
or m := map[string]int{}
.
int, int8/16/32/64, uint… // int is 64-bit on modern platforms; use it by default
float32, float64 // float64 unless you're storing millions of embeddings
string // immutable, UTF-8 bytes
byte = uint8 // a raw byte
rune = int32 // one Unicode code point
bool
[]T, map[K]V, chan T, *T, func(...) ..., interface{ … }, struct{ … }
Go has no implicit conversion, not even int
→ int64
:
var i int = 42
var f float64 = float64(i) // explicit, always
var u uint8 = uint8(300) // ⚠️ silently wraps to 44 — check ranges yourself
n, err := strconv.Atoi("42") // string → int (returns an error!)
s := strconv.Itoa(42) // int → string
f, err := strconv.ParseFloat("0.7", 64)
b, err := strconv.ParseBool("true")
⚠️ string(65)
gives "A"
, not "65"
— it converts a code point. Use strconv
. (go vet
flags this.)
iota
const MaxHistoryTurns = 20 // untyped: adapts to context
const ToolTimeout = 30 * time.Second // typed by inference
type Role string
const (
RoleUser Role = "user"
RoleAssistant Role = "assistant"
RoleSystem Role = "system"
)
type Status int
const (
StatusOK Status = iota // 0 — iota counts from 0 within a const block
StatusRetry // 1
StatusFailed // 2
)
func (s Status) String() string { // makes it print nicely everywhere
switch s {
case StatusOK: return "ok"
case StatusRetry: return "retry"
case StatusFailed: return "failed"
default: return fmt.Sprintf("Status(%d)", int(s))
}
}
A named string type (type Role string
) is Go's enum: the compiler rejects a raw "usr"
typo where a Role
is expected, while JSON marshalling still just works.
Strings are immutable byte slices holding UTF-8. Indexing gives bytes; ranging gives runes.
s := "café"
len(s) // 5 — BYTES, not characters
s[0] // 99 (byte 'c')
for i, r := range s { // i = byte offset, r = rune
fmt.Printf("%d:%c ", i, r) // 0:c 1:a 2:f 3:é
}
utf8.RuneCountInString(s) // 4 — actual character count
[]rune(s)[3] // 'é' — index by character (allocates)
[]byte(s) // copy to a mutable byte slice
The strings
package covers what Python puts on str
:
strings.TrimSpace(" hi \n") // "hi"
strings.ToLower("Calculate 2+2")
strings.Split("a,b,c", ",") // []string{"a","b","c"}
strings.SplitN("calculate 10*5", "calculate", 2)[1] // " 10*5" (maxsplit)
strings.Join([]string{"a", "b"}, ", ") // "a, b"
strings.HasPrefix(name, "tool:") // also HasSuffix, Contains, EqualFold
strings.ReplaceAll(s, "ok", "done")
strings.Fields(" a b ") // ["a","b"] — split on any whitespace
strings.TrimPrefix(path, "docs/") // prefix-safe (not Trim, which is a char set)
strings.Cut("key=value", "=") // "key", "value", true — the modern splitter
Building strings: +=
in a loop is O(n²) and allocates every time. Use a builder:
var b strings.Builder
b.Grow(len(history) * 64) // one allocation if you can estimate
for _, m := range history {
fmt.Fprintf(&b, "%s: %s\n", m.Role, m.Content)
}
prompt := b.String()
fmt
verbs you'll actually use
fmt.Sprintf("%s scored %.2f", name, score) // string, 2-decimal float
fmt.Sprintf("%d/%d tokens", used, limit) // int
fmt.Sprintf("%q", name) // "calculator" — quoted, like Python's !r
fmt.Sprintf("%v", cfg) // default format
fmt.Sprintf("%+v", cfg) // {Name:agent Model:claude-opus-5} ← field names
fmt.Sprintf("%#v", cfg) // Go syntax — best for debugging
fmt.Sprintf("%T", v) // the dynamic type: *main.Agent
fmt.Errorf("run tool %q: %w", name, err) // %w WRAPS an error (see §5)
%q
is your !r
: it makes ""
and " "
visible in logs. %+v
on a struct is the fastest debugging tool in the language.
A slice is a 3-word header: pointer to a backing array, length, capacity. That header is copied on assignment; the array is not.
xs := []string{"a", "b"} // literal
ys := make([]string, 0, 100) // len 0, cap 100 — preallocate when you know the size
ys = append(ys, "x") // append RETURNS a new header; always reassign
len(xs); cap(xs)
xs = append(xs, ys...) // ... spreads a slice (like Python's *)
copy(dst, src) // copies min(len(dst), len(src))
last10 := history[max(0, len(history)-10):] // sliding window (min/max builtins: Go 1.21+)
⚠️ The aliasing trap — slicing shares the backing array:
all := []int{1, 2, 3, 4, 5}
head := all[:3]
head = append(head, 99) // cap allows it → OVERWRITES all[3]
fmt.Println(all) // [1 2 3 99 5]
Fixes: three-index slicing to cap it (all[:3:3]
forces append
to copy), or slices.Clone(head)
.
⚠️ Never keep a small slice of a huge one — the whole backing array stays alive:
snippet := slices.Clone(bigDoc[:100]) // ✅ 100 bytes retained, not 50 MB
The slices
package (Go 1.21+) replaces most hand-written loops:
slices.Contains(tools, "bash")
slices.Sort(scores)
slices.SortFunc(docs, func(a, b Doc) int { return cmp.Compare(b.Score, a.Score) }) // desc
slices.Index(names, "calculator")
slices.Clone(xs); slices.Reverse(xs); slices.Max(scores)
scores := map[string]float64{"calculator": 0.94}
v := scores["missing"] // 0 — no error, zero value
v, ok := scores["missing"] // ✅ the comma-ok idiom: v=0, ok=false
delete(scores, "calculator")
len(scores)
clear(scores) // Go 1.21+
for k, v := range scores { … } // ⚠️ ORDER IS RANDOMIZED, deliberately
keys := slices.Sorted(maps.Keys(scores)) // Go 1.23+ — deterministic iteration
dict.get
vs []
.sync.RWMutex
or use sync.Map
(only for its two specific patterns — see §6.6).make(map[string]int, 1000)
.
type AgentConfig struct {
Name string `json:"name"`
Model string `json:"model"`
Temperature float64 `json:"temperature,omitempty"`
Tools []string `json:"tools,omitempty"`
apiKey string `json:"-"` // lowercase = unexported; "-" = never marshalled
}
cfg := AgentConfig{Name: "researcher", Model: "claude-opus-5"} // ✅ field names, always
p := &cfg // pointer
p.Temperature = 0.2 // auto-dereference — no -> in Go
fmt.Printf("%+v\n", cfg)
Exported = capitalized. Name
is visible outside the package; apiKey
is not. That single rule replaces public
/private
.
Struct tags are metadata read by reflection — the JSON, DB, and validation layers all use them.
Value or pointer?
| Use a value | Use a pointer |
|---|---|
Small, immutable-ish (time.Time , Point ) |
|
| The method mutates the receiver | |
| You want a copy (concurrency safety) | |
| The struct is large (copying costs) | |
| Zero value is meaningful | Nil must be distinguishable from empty |
Go is always pass-by-value — passing a struct copies it; passing a pointer copies the pointer. Slices, maps, and channels contain internal pointers, so copying the header still shares the data.
if err := run(ctx); err != nil { // ✅ init statement scopes err to the if
return fmt.Errorf("run: %w", err)
}
switch { // no condition = cleaner if/else-if chain
case score > 0.9: label = "high"
case score > 0.5: label = "medium"
default: label = "low"
}
switch status { // no fallthrough by default (unlike C)
case StatusOK, StatusRetry: // multiple values per case
continue
}
for i := 0; i < n; i++ { } // classic
for i, msg := range history { } // range: index+value
for _, msg := range history { } // value only
for k := range scores { } // map: keys only
for range 5 { } // Go 1.22+: repeat N times
for { break } // infinite loop — the only `while`
for msg := range ch { } // range over a channel until it's closed
for tok := range stream.Tokens() { } // Go 1.23+: range over an iterator function
There is no while
, no ternary, and no do/while
. That's not an oversight — it's the "one obvious way" principle.
⚠️ range
copies each element: for _, d := range docs { d.Score = 0 }
mutates a copy. Use for i := range docs { docs[i].Score = 0 }
.
✅ Since Go 1.22, loop variables are per-iteration, so the classic "all goroutines see the last value" bug is gone. On older versions you needed i := i
inside the loop.
goto
exists; you will not use it. Labeled break
/continue
are occasionally right for breaking out of nested loops:
outer:
for _, doc := range docs {
for _, chunk := range doc.Chunks {
if chunk.Match(q) { break outer }
}
}
🎯 Actionable rules
- Design types so the zero value is useful; never return a nil map you expect callers to write to.
- Always reassign the result of
append
, andslices.Clone
anything you retain from a big slice.- Use comma-ok on map reads whenever "absent" and "zero" differ.
%+v
and%q
in every debug print;%w
in every wrapped error.
defer
// Summarize returns a summary of text capped at maxWords words.
//
// It collapses whitespace and never splits a word. maxWords must be > 0.
func Summarize(text string, maxWords int) (string, error) {
if maxWords <= 0 {
return "", fmt.Errorf("maxWords must be positive, got %d", maxWords)
}
words := strings.Fields(text)
if len(words) > maxWords {
words = words[:maxWords]
}
return strings.Join(words, " "), nil
}
summary, err := Summarize(doc, 50)
if err != nil { … }
** (T, error) is the signature of Go.** The error is the last return value, always. There is no
Optional
, no exception, no hidden control flow.Doc comments start with the identifier's name and are the package's documentation (go doc
, pkg.go.dev). Exported identifiers without a comment are flagged by linters — and the comment is what an LLM reads when your function becomes a tool.
func splitHostPort(s string) (host string, port int, err error) { // named returns
// … named results are pre-declared and zero-valued; a bare `return` returns them
return host, port, nil // ✅ still return explicitly for clarity
}
Use named returns for documentation and for defer
-based error wrapping (§3.4) — not as an excuse for naked return
s in long functions.
func RunTool(name string, args ...any) (string, error) { … }
RunTool("calculator", "2+2")
RunTool("search", queryArgs...) // spread a slice
type ToolFunc func(ctx context.Context, args json.RawMessage) (string, error)
var registry = map[string]ToolFunc{} // string → behaviour, the Go way
func Register(name string, fn ToolFunc) { registry[name] = fn }
Functions are values: assign them, store them in maps, pass them, return them. That covers most of what Python decorators do.
func makeRetrier(attempts int, base time.Duration) func(context.Context, func() error) error {
return func(ctx context.Context, op func() error) error {
var err error
for i := range attempts {
if err = op(); err == nil {
return nil
}
select {
case <-time.After(base << i): // exponential backoff
case <-ctx.Done():
return ctx.Err()
}
}
return fmt.Errorf("after %d attempts: %w", attempts, err)
}
}
retry := makeRetrier(3, 100*time.Millisecond)
Closures capture variables by reference, so a closure can outlive the function that made it — the compiler moves those variables to the heap (see escape analysis, §7.4).
defer
in practice
defer
schedules a call to run when the surrounding function returns — on any path, including panic. It is Go's with
/finally
.
func fetchDoc(ctx context.Context, url string) ([]byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, fmt.Errorf("fetchDoc: build request: %w", err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, fmt.Errorf("fetchDoc: %w", err)
}
defer resp.Body.Close() // ✅ immediately after the error check, every time
…
}
Four rules that cover every defer
bug:
defer
time
start := time.Now()
defer log.Printf("took %s", time.Since(start)) // ❌ Since() runs NOW → always ~0
defer func() { log.Printf("took %s", time.Since(start)) }() // ✅ closure defers the read
for _, p := range paths {
f, _ := os.Open(p)
defer f.Close() // ❌ 10 000 open files, all closed at the very end
}
for _, p := range paths { // ✅ give each iteration its own function
func() {
f, _ := os.Open(p); defer f.Close(); process(f)
}()
}
func (s *Store) Save(ctx context.Context, d Doc) (err error) {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil { return err }
defer func() {
if err != nil { _ = tx.Rollback(); return }
err = tx.Commit()
}()
…
}
⚠️ Deferred Close()
on a writer can silently drop errors. For files you write, close explicitly and check, or capture it: defer func() { err = errors.Join(err, f.Close()) }()
.
init()
and package-level state
func init() { … } // runs once, after package vars, before main
Use it almost never: it hides work, runs on import, and makes tests order-dependent. Prefer an explicit constructor called from main
. The one defensible use is registering a driver or a codec.
🎯 Actionable rules
- Return
(T, error)
; handle or wrap the error at the very next line.defer
the cleanup on the line after the error check that acquired the resource.- No
defer
inside loops — wrap the body in a function.- Doc-comment every exported identifier, starting with its name.
type Agent struct {
cfg AgentConfig
llm LLMClient
history []Message
mu sync.Mutex
}
// NewAgent constructs an Agent. Constructor functions are Go's __init__.
func NewAgent(cfg AgentConfig, llm LLMClient) (*Agent, error) {
if cfg.Name == "" {
return nil, errors.New("agent: name is required")
}
return &Agent{cfg: cfg, llm: llm}, nil
}
func (a *Agent) AddMessage(role Role, content string) { // pointer receiver: mutates
a.mu.Lock()
defer a.mu.Unlock()
a.history = append(a.history, Message{Role: role, Content: content})
}
func (a *Agent) Len() int { return len(a.history) } // pointer for consistency
func (c AgentConfig) Describe() string { // value receiver: read-only, small
return fmt.Sprintf("%s/%s@%.1f", c.Name, c.Model, c.Temperature)
}
Receiver rules:
sync.Mutex
(copying a mutex is a bug go vet
catches).*T
satisfies an interface when methods have pointer receivers — a plain T
value won't compile. This is the #1 "why doesn't my type implement this interface" error.
type BaseTool struct {
Name string
Description string
}
func (b BaseTool) Schema() string { … }
type CalculatorTool struct {
BaseTool // embedded: no field name
Precision int
}
calc := CalculatorTool{BaseTool: BaseTool{Name: "calculator"}, Precision: 4}
calc.Name // promoted field
calc.Schema() // promoted method
Embedding promotes fields and methods — it looks like inheritance but it's delegation: there is no virtual dispatch and no super
. Embedding an interface is the standard way to build decorators and partial fakes:
type loggingStore struct {
Store // embedded interface: unimplemented methods pass through
log *slog.Logger
}
func (s loggingStore) Get(ctx context.Context, id string) (Doc, error) {
s.log.Info("get", "id", id)
return s.Store.Get(ctx, id)
}
There is no implements
keyword. If the method set matches, the type satisfies the interface.
// Defined in the package that USES it, not the one that implements it.
type LLMClient interface {
Complete(ctx context.Context, prompt string) (string, error)
}
type AnthropicClient struct{ … }
func (c *AnthropicClient) Complete(ctx context.Context, p string) (string, error) { … }
// *AnthropicClient now satisfies LLMClient. No import of your package required.
agent, _ := NewAgent(cfg, &AnthropicClient{}) // prod
agent, _ := NewAgent(cfg, &fakeLLM{reply: "42"}) // test — no mocking library needed
The three rules that make Go interfaces work:
io.Reader
has one method. A 12-method interface is a class in disguise; nobody can fake it in a test.
var _ LLMClient = (*AnthropicClient)(nil) // compile-time assertion that it satisfies
any
, type assertions, and type switches
var v any = payload // any == interface{} (Go 1.18+ alias)
s, ok := v.(string) // ✅ comma-ok: never panics
s := v.(string) // ❌ panics if v isn't a string
switch x := v.(type) { // type switch
case string:
return x
case map[string]any:
return fmt.Sprintf("%d keys", len(x))
case nil:
return "null"
default:
return fmt.Sprintf("unsupported %T", x)
}
any
throws away the compiler's help — use it only at the JSON/reflection boundary and convert into a real type immediately (the same discipline as Python's Any
).
⚠️ The typed-nil trap — an interface holding a nil pointer is not nil:
func newClient() *AnthropicClient { return nil }
var c LLMClient = newClient()
c == nil // false! the interface has a type (*AnthropicClient) and a nil value
Fix: return the interface type as a literal nil
, never a typed nil pointer. Most commonly this bites with error
— never declare var err *MyError
and return it as error
.
Type parameters (Go 1.18+) exist to remove copy-paste, not to build hierarchies.
func Map[T, U any](xs []T, f func(T) U) []U {
out := make([]U, 0, len(xs))
for _, x := range xs {
out = append(out, f(x))
}
return out
}
names := Map(tools, func(t Tool) string { return t.Name() })
func Keys[K comparable, V any](m map[K]V) []K { … } // comparable = usable as a map key
type Number interface{ ~int | ~int64 | ~float64 } // ~ = "any type whose underlying type is"
func Sum[T Number](xs []T) T { var s T; for _, x := range xs { s += x }; return s }
// A generic, type-safe cache — the common real-world use.
type Cache[K comparable, V any] struct {
mu sync.RWMutex
m map[K]V
}
func NewCache[K comparable, V any]() *Cache[K, V] {
return &Cache[K, V]{m: make(map[K]V)}
}
func (c *Cache[K, V]) Get(k K) (V, bool) {
c.mu.RLock(); defer c.mu.RUnlock()
v, ok := c.m[k]
return v, ok
}
When not to use generics: if an interface expresses it, use the interface. Generics can't have methods with their own type parameters, they inflate compile times, and Map
/Filter
chains read worse in Go than a plain for
loop. The slices
, maps
, and cmp
packages already cover 90% of what you'd write.
| Interface | Method | Why it matters |
|---|---|---|
error |
||
Error() string |
||
| Every failure (§5) | ||
fmt.Stringer |
||
String() string |
||
Custom formatting in every %v |
||
io.Reader / io.Writer |
||
Read /Write |
||
| Files, sockets, buffers, HTTP bodies — all compose | ||
io.Closer |
||
Close() error |
||
Pairs with defer |
||
json.Marshaler / Unmarshaler |
||
| Custom JSON | Enums, time formats, LLM payload quirks | |
context.Context |
||
Done , Err , Value , Deadline |
||
| Cancellation everywhere (§6.5) | ||
http.Handler |
||
ServeHTTP |
||
| Every middleware in Go | ||
sort.Interface |
||
Len /Less /Swap |
||
Mostly superseded by slices.SortFunc |
||
io.Reader
/io.Writer
are the reason Go plumbing composes so well: an HTTP body, a gzip stream, a file, and a bytes.Buffer
are interchangeable.
🎯 Actionable rules
- Constructors return
(*T, error)
; validate there, so an existing value is always valid.- Define small interfaces in the consuming package; accept interfaces, return structs.
var _ Iface = (*T)(nil)
to assert satisfaction at compile time.- Reach for generics only after you've written the same function twice.
type error interface {
Error() string
}
That's it. An error is any value with an Error() string
method. There is no stack unwinding, no exception hierarchy, no invisible control flow — which is why Go code has if err != nil
everywhere and why you can always see the failure path.
errors.New("agent: name is required") // static message
fmt.Errorf("embed batch %d: %w", i, err) // wrap with context
fmt.Errorf("parse config: %v", err) // %v = context WITHOUT wrapping
errors.Join(err1, err2) // multiple failures (Go 1.20+)
%w
vs %v
:%w
keeps the original error reachable by errors.Is
/errors.As
; %v
flattens it to text. Wrap by default; use %v
deliberately when you don't want callers coupling to an internal error type.
Follow one convention across the codebase — this repo's ( CLAUDE.md) is
fmt.Errorf("packagename.FuncName: %w", err)
:
func (r *Repo) GetDoc(ctx context.Context, id string) (Doc, error) {
var d Doc
if err := r.db.GetContext(ctx, &d, qGetDoc, id); err != nil {
return Doc{}, fmt.Errorf("repo.GetDoc: %w", err)
}
return d, nil
}
Read top-to-bottom, the final message becomes a trace:
handler.Query: service.Answer: repo.GetDoc: sql: no rows in result set
Rules: add context, not restatement (never "error: %w"
); don't capitalize or end with punctuation; never log and return the same error — pick one, and log at the boundary that handles it.
Is
, As
// Sentinel: a comparable, exported value callers can test for.
var (
ErrNotFound = errors.New("not found")
ErrRateLimit = errors.New("rate limited")
)
// Custom type: when the caller needs structured detail.
type ToolError struct {
Tool string
Code int
Err error
}
func (e *ToolError) Error() string { return fmt.Sprintf("tool %s: %v", e.Tool, e.Err) }
func (e *ToolError) Unwrap() error { return e.Err } // makes errors.Is see through it
// Callers:
if errors.Is(err, ErrNotFound) { // ✅ works through any wrapping
return http.StatusNotFound, nil
}
var toolErr *ToolError
if errors.As(err, &toolErr) { // ✅ extract the typed error
metrics.ToolFailures.WithLabelValues(toolErr.Tool).Inc()
}
if err == ErrNotFound { } // ❌ breaks the moment someone wraps
errors.Is
for identity, errors.As
for structure. Never compare error strings.
// ✅ Handle immediately; the happy path stays at the left margin.
resp, err := c.Complete(ctx, prompt)
if err != nil {
return fmt.Errorf("agent.Run: %w", err)
}
use(resp)
// ✅ Retry only what's retryable.
for attempt := range maxAttempts {
out, err = call(ctx)
if err == nil { break }
if !errors.Is(err, ErrRateLimit) && !isTransient(err) {
return fmt.Errorf("agent.call: %w", err) // permanent → stop immediately
}
select {
case <-time.After(backoff(attempt)):
case <-ctx.Done():
return ctx.Err()
}
}
// ✅ Deliberately ignoring an error is written, not implied.
_ = resp.Body.Close()
defer func() { _ = tx.Rollback() }() // rollback after a commit is a no-op
js
// ✅ Collect failures across a batch instead of stopping at the first.
var errs []error
for _, chunk := range chunks {
if err := index(ctx, chunk); err != nil {
errs = append(errs, fmt.Errorf("chunk %s: %w", chunk.ID, err))
}
}
return errors.Join(errs...) // nil if the slice is empty
panic
unwinds the goroutine and crashes the process unless recovered. It is not an exception system.
Panic only when the program cannot sensibly continue: an impossible invariant, a programming bug, or failed initialization at startup (regexp.MustCompile
, template.Must
— the Must
prefix is the convention).
Recover only at a process boundary — one bad request must not kill the server:
func Recoverer(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
if rec := recover(); rec != nil {
slog.Error("panic in handler",
"err", rec, "path", r.URL.Path, "stack", string(debug.Stack()))
http.Error(w, "internal error", http.StatusInternalServerError)
}
}()
next.ServeHTTP(w, r)
})
}
⚠️ ** recover only works in the same goroutine.** A panic inside
go func(){…}()
kills the whole process no matter what your HTTP middleware does — every goroutine you spawn needs its own recover, or must be provably panic-free.| Python | Go |
|---|---|
raise ValueError("bad temp") |
return fmt.Errorf("bad temperature %v", t) |
except ValueError: |
if errors.Is(err, ErrBadTemp) |
except SomeError as e: e.field |
var e *SomeError; errors.As(err, &e) |
raise X from err |
fmt.Errorf("context: %w", err) |
finally: |
defer |
except Exception: pass |
_ = f() (and a comment saying why) |
| Traceback | The wrap chain you built by hand |
sys.exit(1) on fatal config |
log.Fatal / panic in main only |
🎯 Actionable rules
- Wrap with
%w
and apkg.Func:
prefix at every layer; log once, at the top.errors.Is
for sentinels,errors.As
for typed detail — never string comparison.- Panic only for programmer bugs and startup failures; recover only at boundaries.
- Every goroutine you start needs its own panic protection.
Go's headline feature. It is also where every serious Go bug lives.
go doWork() // that's the entire syntax
go func(id string) { … }(docID) // pass arguments explicitly
A goroutine is a user-space thread multiplexed onto OS threads by the Go runtime: ~2 KB of initial stack (grown on demand), microsecond creation. A hundred thousand of them in one process is normal; a hundred thousand OS threads is not.
The rule that prevents most production incidents: never start a goroutine without knowing how it stops. Every goroutine needs an exit condition — a closed channel, a cancelled context, or a finite loop. A goroutine blocked forever on a channel nobody writes to is a leak: its stack, its captured variables, and everything they reference stay alive until the process dies.
// ❌ leaks one goroutine per request, forever, if nobody reads results
go func() { results <- expensive() }()
// ✅ it can always exit
go func() {
select {
case results <- expensive():
case <-ctx.Done():
}
}()
A channel is a typed, concurrency-safe queue. Unbuffered channels are a rendezvous: the sender blocks until a receiver takes the value.
ch := make(chan Token) // unbuffered: synchronous handoff
buf := make(chan Job, 100) // buffered: sender proceeds until full
ch <- tok // send
tok := <-ch // receive
tok, ok := <-ch // ok == false when the channel is closed AND drained
close(ch) // only the SENDER closes, and only once
for tok := range ch { … } // receives until closed
Directional types document intent and are checked by the compiler:
func produce(out chan<- Token) { … } // send-only
func consume(in <-chan Token) { … } // receive-only
| Operation | On a nil channel | On a closed channel |
|---|---|---|
| Send | blocks forever | panics |
| Receive | blocks forever | returns zero value immediately, ok=false |
| Close | panics | panics |
Consequences: only ever close from the single owning sender; closing signals "no more values", not "stop". To stop a consumer, cancel its context.
select
select {
case tok := <-tokens:
emit(tok)
case err := <-errs:
return err
case <-ctx.Done(): // cancellation, always include it
return ctx.Err()
case <-time.After(5 * time.Second): // per-iteration timeout
return errors.New("stream stalled")
default: // non-blocking: runs if nothing else is ready
metrics.Idle.Inc()
}
select
blocks until one case is ready, choosing randomly among ready cases. With default
it never blocks. ⚠️ time.After
allocates a timer per call — inside a hot loop use a reusable time.NewTimer
/Ticker
and stop it.
1. Bounded worker pool — N workers over a job channel. The default for embedding, indexing, or crawling:
func EmbedAll(ctx context.Context, chunks []string, workers int) ([][]float32, error) {
type result struct {
i int
vec []float32
err error
}
jobs := make(chan int)
out := make(chan result, len(chunks))
var wg sync.WaitGroup
for range workers { // fixed number of goroutines
wg.Add(1)
go func() {
defer wg.Done()
for i := range jobs { // exits when jobs is closed
v, err := embed(ctx, chunks[i])
out <- result{i, v, err}
}
}()
}
go func() { // feed, then close so workers exit
defer close(jobs)
for i := range chunks {
select {
case jobs <- i:
case <-ctx.Done():
return
}
}
}()
wg.Wait()
close(out)
vecs := make([][]float32, len(chunks))
for r := range out {
if r.err != nil {
return nil, fmt.Errorf("embed chunk %d: %w", r.i, r.err)
}
vecs[r.i] = r.vec // index carries the order back
}
return vecs, nil
}
2. errgroup — the concise version when you just need "run these, stop on first error":
import "golang.org/x/sync/errgroup"
g, ctx := errgroup.WithContext(ctx) // ctx is cancelled as soon as one task fails
g.SetLimit(8) // ← bounded concurrency, one line
results := make([]Doc, len(ids))
for i, id := range ids {
g.Go(func() error { // Go 1.22+: no `i := i` needed
d, err := fetch(ctx, id)
if err != nil {
return fmt.Errorf("fetch %s: %w", id, err)
}
results[i] = d // ✅ distinct indices — no mutex required
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
This is Go's asyncio.gather
Semaphore
, with cancellation included.
3. Pipeline / fan-in — merge several streams into one, the shape behind multi-model or multi-tool streaming:
func merge[T any](ctx context.Context, chans ...<-chan T) <-chan T {
out := make(chan T)
var wg sync.WaitGroup
for _, c := range chans {
wg.Add(1)
go func(c <-chan T) {
defer wg.Done()
for v := range c {
select {
case out <- v:
case <-ctx.Done():
return
}
}
}(c)
}
go func() { wg.Wait(); close(out) }() // close exactly once, after all senders finish
return out
}
context
: cancellation that actually propagates
context.Context
carries a deadline, a cancellation signal, and request-scoped values down the call tree. Every function that does I/O takes one as its first parameter.
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel() // ✅ ALWAYS defer cancel — otherwise the timer leaks
resp, err := agent.Run(ctx, prompt)
switch {
case errors.Is(err, context.DeadlineExceeded):
http.Error(w, "upstream timeout", http.StatusGatewayTimeout)
case errors.Is(err, context.Canceled):
return // client hung up; nothing to write
}
Why it matters for AI services: when a user closes the browser mid-stream, r.Context()
is cancelled, and that cancellation flows into your model call, your DB query, and every worker goroutine — so you stop paying for tokens nobody will read.
// Values: request-scoped metadata only, with an unexported key type.
type ctxKey struct{}
var tenantKey ctxKey
ctx = context.WithValue(ctx, tenantKey, tenant)
tenant, ok := ctx.Value(tenantKey).(string)
Rules: ctx
is the first parameter, never stored in a struct; context.Background()
only in main
/tests; never pass nil
; values are for tracing/tenancy, never for optional arguments.
sync
: when channels are overkill "Don't communicate by sharing memory; share memory by communicating." …but a mutex around a cache is simpler than a channel, and simpler wins.
type Cache struct {
mu sync.RWMutex // zero value is ready — no initialization
m map[string][]float32
}
func (c *Cache) Get(k string) ([]float32, bool) {
c.mu.RLock() // many concurrent readers
defer c.mu.RUnlock()
v, ok := c.m[k]
return v, ok
}
func (c *Cache) Put(k string, v []float32) {
c.mu.Lock() // one writer, excludes readers
defer c.mu.Unlock()
c.m[k] = v
}
var once sync.Once
once.Do(func() { tokenizer = loadTokenizer() }) // exactly-once init
var wg sync.WaitGroup // wg.Add before `go`, wg.Done in a defer
var inflight atomic.Int64 // lock-free counters
inflight.Add(1); defer inflight.Add(-1)
Use sync.Map
only for its two documented patterns (write-once/read-many, or disjoint key sets per goroutine); otherwise a plain map with an RWMutex
is faster and clearer. Put the mutex next to the data it protects, and document what it guards.
go test -race ./...
go run -race ./cmd/api
It catches unsynchronized concurrent access at runtime (~10× slower, more memory — fine for CI). A data race in Go is undefined behaviour, not just a wrong number: a torn map write crashes the process.
| Symptom | Cause | Fix |
|---|---|---|
| Memory grows forever | Goroutine leak — blocked send/receive | Add <-ctx.Done() to every select ; close channels |
all goroutines are asleep - deadlock! |
||
Unbuffered send with no receiver; wg.Wait() before Done |
||
Check ownership; wg.Add before go |
||
send on closed channel panic |
||
| Multiple senders, or closing to signal "stop" | Only the sole sender closes; cancel via context | |
| Results in the wrong order | Concurrency doesn't preserve order | Carry an index, or write into a preallocated slice |
| Rare corrupt data | Data race | |
-race , then a mutex or channel |
||
| 429s / OOM under load | Unbounded fan-out | |
g.SetLimit(n) or a worker pool |
||
context deadline exceeded everywhere |
||
| One deadline shared by N sequential calls | Give each call its own budget |
🎯 Actionable rules
- Every goroutine has a known exit path; every blocking
select
has<-ctx.Done()
.- Bound concurrency explicitly —
errgroup.SetLimit
or a fixed worker pool. Nevergo
in an unbounded loop.ctx
first parameter,defer cancel()
always.- Run
-race
in CI, permanently.
You don't have to know this to write Go. You do have to know it to explain a p99 latency spike.
G = goroutine M = OS thread P = processor (a scheduling context, GOMAXPROCS of them)
[P0]──local run queue──> G G G each P owns a queue of runnable Gs
[P1]──local run queue──> G an idle P steals work from a busy one
↑ bound to an M (thread) while running
[global run queue] ── overflow ──
GOMAXPROCS
go.uber.org/automaxprocs
) or your 500m-CPU pod will spawn 64 Ps and thrash.async
/await
colouring.Versus Python: asyncio
gives you one thread cooperatively multiplexing coroutines, and any blocking call freezes all of them. Go gives you preemptive scheduling across every core with no code-colour distinction. That's the core reason a Go gateway holds 50k streaming connections on hardware where a Python one needs process fan-out.
Go's GC is a concurrent, tri-colour mark-and-sweep collector, non-generational and non-compacting. It's tuned for latency, not throughput: sub-millisecond stop-the-world s, at the cost of some CPU and headroom.
GOGC=100 # default: collect when the heap doubles since the last GC
GOGC=200 # collect half as often — more RAM, less CPU
GOMEMLIMIT=6GiB # soft memory ceiling (Go 1.19+) — the setting for containers
GODEBUG=gctrace=1 ./api # one line per GC cycle: heap size, , CPU share
In containers, set GOMEMLIMIT to ~80% of the pod's memory limit. Without it, Go sizes the heap from
GOGC
alone, happily grows past the cgroup limit, and gets OOM-killed with no Go-level error. With it, the GC works harder as you approach the ceiling instead of dying.Pointer-heavy structures make GC scan more. Fewer, larger allocations of pointer-free data ([]float32
for embeddings, not []*float32
) is the single biggest GC win in AI workloads.
A write in one goroutine is only guaranteed visible to another if they synchronize — via a channel operation, a mutex, sync/atomic
, sync.Once
, or WaitGroup
. Without that, the compiler and CPU may reorder freely, and the race detector will (eventually) tell you. There is no "volatile"; there is sync/atomic
.
The compiler puts values on the stack (free, no GC) unless they can outlive the function, in which case they escape to the heap.
go build -gcflags='-m' ./... # prints "escapes to heap" / "does not escape"
Common causes of escape: returning a pointer to a local, storing in an interface, closing over a variable, sending on a channel, fmt.Sprintf
.
Allocation-reduction techniques, in order of payoff:
out := make([]Doc, 0, len(ids)) // 1. preallocate with capacity — avoids log(n) regrowths
m := make(map[string]int, 1000)
var b strings.Builder // 2. builders instead of += concatenation
b.Grow(estimate)
var bufPool = sync.Pool{ // 3. pool big, short-lived buffers on hot paths
New: func() any { return new(bytes.Buffer) },
}
buf := bufPool.Get().(*bytes.Buffer)
defer func() { buf.Reset(); bufPool.Put(buf) }()
func (s *Scanner) Fill(dst []byte) int // 4. let the caller own the buffer
Do these where a profile says they matter (§12), not everywhere. sync.Pool
used carelessly is a memory leak with extra steps.
| Workload | Winner | Why |
|---|---|---|
| 20k concurrent SSE streams | ||
| Go, decisively | ||
| 2 KB goroutines vs event-loop + process fan-out | ||
| Fan-out to 50 tools/APIs per request | Go | |
errgroup + real parallelism |
||
| JSON/protobuf transformation at volume | Go | |
| Compiled, GC-friendly, no interpreter overhead | ||
| Token/rate accounting, queues, schedulers | Go | |
| Predictable latency, cheap primitives | ||
| Embedding, training, fine-tuning | Python | |
| torch/numpy/CUDA live there | ||
| Data science, notebooks, evaluation | Python | |
| The ecosystem is the product | ||
| Model-specific pre/post-processing | Python | |
| Tokenizers and libraries exist already |
🎯 Actionable rules
- In containers: set
GOMEMLIMIT
(~80% of the limit) and makeGOMAXPROCS
cgroup-aware.- Preallocate slices and maps whose size you know.
- Prefer pointer-free bulk data (
[]float32
) to reduce GC scan time.- Optimize allocations only where a pprof profile points.
Go's stdlib is unusually complete: an HTTP/2 server, JSON, TLS, templating, profiling, and testing all ship with the compiler. The list below is what an AI service actually uses.
net/http
— the server
mux := http.NewServeMux()
mux.HandleFunc("POST /v1/query", h.Query) // Go 1.22+: method + wildcards
mux.HandleFunc("GET /v1/jobs/{id}", h.GetJob) // r.PathValue("id")
srv := &http.Server{
Addr: ":8080",
Handler: Recoverer(RequestID(Logging(mux))), // middleware = wrapped handlers
ReadHeaderTimeout: 5 * time.Second, // ✅ blocks Slowloris; the one people forget
ReadTimeout: 30 * time.Second,
WriteTimeout: 0, // 0 for SSE/streaming endpoints; set it otherwise
IdleTimeout: 120 * time.Second,
MaxHeaderBytes: 1 << 20,
}
// Graceful shutdown: stop accepting, let in-flight requests finish.
go func() {
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
slog.Error("listen", "err", err); os.Exit(1)
}
}()
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
<-ctx.Done()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
_ = srv.Shutdown(shutdownCtx)
chi
adds routers, groups, and middleware chains on top of http.Handler
without inventing a new handler type — which is why it composes with everything (and why this repo uses it).
net/http
— the client
var client = &http.Client{ // ✅ ONE client for the process, reused
Timeout: 60 * time.Second, // total budget, including body read
Transport: &http.Transport{
MaxIdleConns: 200,
MaxIdleConnsPerHost: 100, // default is 2 — far too low for an LLM proxy
IdleConnTimeout: 90 * time.Second,
},
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
if err != nil { return fmt.Errorf("llm.Complete: %w", err) }
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil { return fmt.Errorf("llm.Complete: %w", err) }
defer resp.Body.Close() // ✅ ALWAYS — otherwise the connection leaks
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<10)) // cap what you read on errors
return fmt.Errorf("llm.Complete: status %d: %s", resp.StatusCode, b)
}
Three non-negotiables: reuse the client, always close the body, always pass a context. Creating an http.Client
per request disables connection pooling and exhausts sockets under load.
encoding/json
type QueryIn struct {
Query string `json:"query"`
Temperature float64 `json:"temperature,omitempty"` // omit when zero
Tools []string `json:"tools,omitempty"`
internal string `json:"-"` // never marshalled
}
b, err := json.Marshal(v)
err = json.Unmarshal(b, &v) // note the pointer
dec := json.NewDecoder(r.Body) // ✅ stream, don't ReadAll
dec.DisallowUnknownFields() // ✅ typo'd client fields become errors
if err := dec.Decode(&in); err != nil {
http.Error(w, "invalid body", http.StatusBadRequest); return
}
var raw json.RawMessage // defer parsing tool args
enc := json.NewEncoder(w); enc.Encode(out) // stream the response out
⚠️ Only exported fields are marshalled. ⚠️ Unmarshalling into map[string]any
turns every number into float64
— decode into a struct whenever you can. For hot paths, json.Decoder
on the body avoids materializing the whole payload.
Custom marshalling for domain types:
func (r Role) MarshalJSON() ([]byte, error) { return json.Marshal(string(r)) }
log/slog
— structured logging (Go 1.21+)
logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo}))
slog.SetDefault(logger)
slog.Info("tool completed", "tool", name, "ms", elapsed.Milliseconds(), "tokens", n)
slog.Error("model call failed", "err", err, "model", cfg.Model, "attempt", i)
reqLog := logger.With("request_id", rid, "tenant", tenant) // bind once, reuse per request
reqLog.Info("received")
Structured key-value output is what makes logs queryable in Loki/Datadog. Never log prompts, keys, or full request bodies — log ids, counts, durations, and truncated previews.
time
time.Now(); time.Since(start) // monotonic for durations
30 * time.Second; 500 * time.Millisecond // Durations are typed ints — no unit bugs
t.Format(time.RFC3339); time.Parse(time.RFC3339, s)
time.Now().UTC() // store UTC, convert at the edge
tick := time.NewTicker(10 * time.Second)
defer tick.Stop() // ✅ tickers leak if not stopped
select {
case <-tick.C: flushMetrics()
case <-ctx.Done(): return
}
io
and bufio
— the composable plumbing
io.Copy(dst, src) // stream, constant memory
io.ReadAll(io.LimitReader(r, 10<<20)) // ✅ always cap untrusted input
io.MultiWriter(w, &buf) // tee the response into a buffer
sc := bufio.NewScanner(resp.Body) // line-by-line: perfect for SSE
sc.Buffer(make([]byte, 0, 64*1024), 1<<20) // ✅ raise the 64 KB line limit
for sc.Scan() {
line := sc.Text()
…
}
if err := sc.Err(); err != nil { … } // ✅ Scan() returning false isn't always EOF
| Package | Use it for |
|---|---|
context |
|
| Cancellation and deadlines (§6.5) | |
sync / sync/atomic |
|
Mutexes, WaitGroup , Once , counters (§6.6) |
|
errors |
|
Is , As , Join , Unwrap (§5) |
|
strconv / strings / bytes |
|
| Conversion and text handling (§2.4) | |
regexp |
|
RE2 — linear time, no catastrophic backtracking; MustCompile at package level |
|
os / os/signal |
|
| Env, files, SIGTERM handling | |
flag |
|
Small CLIs; use cobra for a command tree |
|
embed |
|
//go:embed prompts/*.md — bake prompts and migrations into the binary |
|
text/template |
|
| Prompt templating with named fields | |
database/sql (+ sqlx , pgx ) |
|
SQL; always QueryContext , always defer rows.Close() , always check rows.Err() |
|
encoding/base64 , crypto/* |
|
Tokens, signatures, crypto/rand for secrets |
|
net/http/httptest |
|
| In-process HTTP tests (§10) | |
runtime/pprof , net/http/pprof |
|
| Profiling (§12) | |
testing |
|
| Tests, benchmarks, fuzzing — all built in |
Third-party worth adopting: golang.org/x/sync/errgroup
and singleflight
, go-chi/chi
, jmoiron/sqlx
, stretchr/testify/require
, pressly/goose
, golang.org/x/time/rate
, and OpenTelemetry for traces. Go culture keeps dependency trees small — prefer the stdlib until it genuinely hurts.
🎯 Actionable rules
- One
http.Client
per process with a timeout and a tuned transport;defer resp.Body.Close()
always.- Explicit
http.Server
timeouts and graceful shutdown on SIGTERM.json.Decoder
+DisallowUnknownFields
on request bodies;io.LimitReader
on anything untrusted.slog
with key-value pairs from day one — retrofitting structure is miserable.
What Go is actually for in an AI stack: the request path, the fan-out, and the streaming.
func (c *LLM) Stream(ctx context.Context, prompt string, out chan<- string) error {
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, c.url, encode(prompt))
req.Header.Set("Accept", "text/event-stream")
resp, err := c.http.Do(req)
if err != nil {
return fmt.Errorf("llm.Stream: %w", err)
}
defer resp.Body.Close()
sc := bufio.NewScanner(resp.Body)
sc.Buffer(make([]byte, 0, 64*1024), 1<<20) // model chunks exceed the 64 KB default
for sc.Scan() {
line, ok := strings.CutPrefix(sc.Text(), "data: ")
if !ok || line == "" {
continue
}
if line == "[DONE]" {
return nil
}
var ev struct {
Delta struct{ Text string } `json:"delta"`
}
if err := json.Unmarshal([]byte(line), &ev); err != nil {
return fmt.Errorf("llm.Stream: decode %q: %w", truncate(line, 80), err)
}
select {
case out <- ev.Delta.Text:
case <-ctx.Done(): // client disconnected: stop paying for tokens
return ctx.Err()
}
}
return sc.Err()
}
func (h *Handler) Stream(w http.ResponseWriter, r *http.Request) {
rc := http.NewResponseController(w) // Go 1.20+; replaces the http.Flusher cast
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("X-Accel-Buffering", "no") // stop nginx from buffering your stream
ctx := r.Context() // cancelled when the client goes away
tokens := make(chan string, 16)
errc := make(chan error, 1)
go func() { errc <- h.llm.Stream(ctx, r.FormValue("q"), tokens); close(tokens) }()
for {
select {
case tok, ok := <-tokens:
if !ok {
fmt.Fprint(w, "data: [DONE]\n\n")
_ = rc.Flush()
return
}
fmt.Fprintf(w, "data: %s\n\n", tok)
_ = rc.Flush() // ✅ without Flush nothing reaches the client
case <-ctx.Done():
return
case <-time.After(30 * time.Second):
slog.Warn("stream stalled", "path", r.URL.Path)
return
}
}
}
Remember to set WriteTimeout: 0
on the server for streaming routes (§8.1), or the connection dies mid-answer.
type Tool struct {
Name string `json:"name"`
Description string `json:"description"`
Schema json.RawMessage `json:"input_schema"` // sent verbatim to the model
Run func(ctx context.Context, args json.RawMessage) (string, error) `json:"-"`
}
type Registry struct {
mu sync.RWMutex
tools map[string]Tool
}
func (r *Registry) Register(t Tool) error {
r.mu.Lock(); defer r.mu.Unlock()
if _, dup := r.tools[t.Name]; dup {
return fmt.Errorf("registry.Register: duplicate tool %q", t.Name)
}
r.tools[t.Name] = t
return nil
}
func (r *Registry) Dispatch(ctx context.Context, name string, args json.RawMessage) (string, error) {
r.mu.RLock(); t, ok := r.tools[name]; r.mu.RUnlock()
if !ok {
return "", fmt.Errorf("registry.Dispatch: unknown tool %q", name) // never trust the model
}
ctx, cancel := context.WithTimeout(ctx, 30*time.Second) // ✅ per-tool budget
defer cancel()
return t.Run(ctx, args)
}
Two things the model must never control: which tools exist, and how long they may run.
import "golang.org/x/time/rate"
type Client struct {
http *http.Client
limiter *rate.Limiter // rate.NewLimiter(rate.Limit(50), 100) → 50 rps, burst 100
sem chan struct{} // concurrency cap: make(chan struct{}, 16)
}
func (c *Client) Complete(ctx context.Context, prompt string) (string, error) {
if err := c.limiter.Wait(ctx); err != nil { // blocks or returns on cancellation
return "", fmt.Errorf("llm.Complete: rate wait: %w", err)
}
select { // bound in-flight requests
case c.sem <- struct{}{}:
defer func() { <-c.sem }()
case <-ctx.Done():
return "", ctx.Err()
}
var lastErr error
for attempt := range 4 {
out, err := c.do(ctx, prompt)
if err == nil {
return out, nil
}
lastErr = err
var re *RetryableError
if !errors.As(err, &re) {
return "", fmt.Errorf("llm.Complete: %w", err) // permanent → stop
}
delay := re.RetryAfter // honour the server's hint
if delay == 0 {
delay = time.Duration(1<<attempt) * 200 * time.Millisecond
}
jitter := time.Duration(rand.Int64N(int64(delay / 2))) // math/rand/v2
select {
case <-time.After(delay + jitter):
case <-ctx.Done():
return "", ctx.Err()
}
}
return "", fmt.Errorf("llm.Complete: exhausted retries: %w", lastErr)
}
// Go owns HTTP, auth, tenancy, and fan-out; Python owns the model work.
func (s *Service) Answer(ctx context.Context, tenant, q string) (Answer, error) {
ctx, cancel := context.WithTimeout(ctx, 45*time.Second)
defer cancel()
g, gctx := errgroup.WithContext(ctx)
var (
docs []Doc
vec []float32
)
g.Go(func() (err error) { docs, err = s.repo.Search(gctx, tenant, q); return })
g.Go(func() (err error) { vec, err = s.python.Embed(gctx, q); return }) // internal REST
if err := g.Wait(); err != nil {
return Answer{}, fmt.Errorf("service.Answer: %w", err)
}
…
}
Retrieval and embedding run in parallel; either failure cancels the other; the whole request shares one deadline. That is ~15 lines of Go for what needs careful orchestration elsewhere.
singleflight
— collapse duplicate work When 500 users ask the same question in the same second, do the expensive thing once:
import "golang.org/x/sync/singleflight"
var group singleflight.Group
func (c *Cache) Embed(ctx context.Context, text string) ([]float32, error) {
key := hash(text)
if v, ok := c.Get(key); ok {
return v, nil
}
v, err, _ := group.Do(key, func() (any, error) { // concurrent callers share one result
return c.upstream.Embed(ctx, text)
})
if err != nil {
return nil, fmt.Errorf("cache.Embed: %w", err)
}
return v.([]float32), nil
}
🎯 Actionable rules
- Propagate
r.Context()
into every model call so a disconnect stops the spend.- Bound everything: rate limiter, concurrency semaphore, per-tool timeout, retry cap.
- Flush after every SSE write, and disable proxy buffering.
- Validate tool names against the registry — the model's output is untrusted input.
Testing is in the standard library, in the same package, with no framework to choose. That's a feature.
// internal/service/summarize_test.go
package service
func TestSummarize(t *testing.T) {
tests := []struct {
name string
text string
maxWords int
want string
wantErr bool
}{
{name: "truncates", text: "a b c d", maxWords: 2, want: "a b"},
{name: "collapses whitespace", text: " a b ", maxWords: 5, want: "a b"},
{name: "rejects zero", text: "a", maxWords: 0, wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { // a named subtest per case
t.Parallel() // ✅ subtests run concurrently
got, err := Summarize(tt.text, tt.maxWords)
if tt.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
require.Equal(t, tt.want, got)
})
}
}
go test
failures print the subtest path (TestSummarize/rejects_zero
), so you know exactly which case broke. Per this repo's conventions, use testify/require
(stops the test) over assert
(continues) and over bare t.Fatal
.
Helpers that pay for themselves:
t.Helper() // in a helper: failures report the CALLER's line
t.Cleanup(func() { … }) // teardown, LIFO, runs even on failure — better than defer
t.TempDir() // auto-removed temp directory
t.Setenv("MODEL", "x") // auto-restored env (forbids t.Parallel in that test)
t.Context() // Go 1.24+: a context cancelled at test end
testing.Short() // skip slow tests under `go test -short`
Because interfaces are structural and defined by the consumer, a fake is just a struct:
type fakeLLM struct {
replies []string
calls int
}
func (f *fakeLLM) Complete(ctx context.Context, prompt string) (string, error) {
if f.calls >= len(f.replies) {
return "", errors.New("fakeLLM: out of replies")
}
f.calls++
return f.replies[f.calls-1], nil
}
func TestAgentUsesCalculator(t *testing.T) {
llm := &fakeLLM{replies: []string{`{"tool":"calculator","args":{"expression":"10*5"}}`}}
agent, err := NewAgent(AgentConfig{Name: "t"}, llm)
require.NoError(t, err)
resp, err := agent.Run(t.Context(), "calculate 10 * 5")
require.NoError(t, err)
require.Equal(t, StatusOK, resp.Status)
require.Equal(t, 1, llm.calls)
}
No mocking library, no code generation, no patching. If faking your interface is painful, the interface is too big.
httptest
// Test a handler without a network.
func TestQueryHandler(t *testing.T) {
h := NewHandler(&fakeService{})
req := httptest.NewRequest(http.MethodPost, "/v1/query", strings.NewReader(`{"query":"hi"}`))
rec := httptest.NewRecorder()
h.Query(rec, req)
require.Equal(t, http.StatusOK, rec.Code)
require.JSONEq(t, `{"answer":"hi!"}`, rec.Body.String())
}
// Stub an upstream provider with a real server.
func TestClientRetriesOn429(t *testing.T) {
var hits atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if hits.Add(1) < 3 {
w.WriteHeader(http.StatusTooManyRequests)
return
}
_, _ = io.WriteString(w, `{"content":"ok"}`)
}))
defer srv.Close()
c := NewClient(srv.URL)
out, err := c.Complete(t.Context(), "hi")
require.NoError(t, err)
require.Equal(t, "ok", out)
require.EqualValues(t, 3, hits.Load())
}
js
var update = flag.Bool("update", false, "update golden files")
func TestPromptRendering(t *testing.T) {
got := RenderSystemPrompt(cfg)
golden := filepath.Join("testdata", "system_prompt.golden")
if *update {
require.NoError(t, os.WriteFile(golden, []byte(got), 0o644))
}
want, err := os.ReadFile(golden) // testdata/ is ignored by the go tool
require.NoError(t, err)
require.Equal(t, string(want), got)
}
go test ./... -update
regenerates; the diff shows up in code review. Ideal for prompts, schemas, and serialized payloads.
//go:build integration
package repo_test
// … tests that need a real Postgres (testcontainers-go), run with:
// go test -tags integration ./...
Fast unit tests in pre-commit, tagged integration tests in CI — the split this repo's CLAUDE.md prescribes.
func BenchmarkChunk(b *testing.B) {
doc := strings.Repeat("word ", 100_000)
b.ReportAllocs()
b.ResetTimer()
for b.Loop() { // Go 1.24+; older: for i := 0; i < b.N; i++
sink = Chunk(doc, 1000, 200)
}
}
var sink []string // package-level: stops the compiler optimizing the call away
go test -bench=Chunk -benchmem -count=10 ./internal/text | tee new.txt
benchstat old.txt new.txt # statistically meaningful comparison, not one lucky run
-benchmem
prints B/op
and allocs/op
— usually more actionable than ns/op, because allocations drive GC pressure.
func FuzzParseToolCall(f *testing.F) {
f.Add(`{"tool":"calc","args":{}}`) // seed corpus
f.Fuzz(func(t *testing.T, s string) {
_, _ = ParseToolCall([]byte(s)) // must never panic on any input
})
}
go test -fuzz=FuzzParseToolCall -fuzztime=60s ./internal/agent
Anything that parses model output is a prime fuzz target: LLMs emit truncated JSON, nested fences, and 10 MB of whitespace. Crashes land in testdata/fuzz/
and become permanent regression tests.
go test ./... # everything
go test -race ./... # ✅ what CI must run
go test -run TestAgent/calculator # by name, subtests included
go test -short ./... # skip the slow ones
go test -cover ./... && go tool cover -html=cover.out
go test -count=1 ./... # bypass the test cache
🎯 Actionable rules
- Table-driven +
t.Run
+t.Parallel
as the default shape.- Hand-written fakes over mock frameworks; keep interfaces small enough to fake.
-race
in CI, always; fuzz anything that parses untrusted or model-generated input.- Benchmark with
-benchmem
and compare withbenchstat
, never by eyeballing one run.
go mod init github.com/acme/agent-service
go get github.com/go-chi/chi/v5@latest
go get -u ./... # update dependencies
go mod tidy # add what's used, drop what isn't — run before every commit
go mod download # populate the module cache (Docker builds)
go mod why github.com/x/y # who pulled this in?
go work init ./backend-go ./shared # multi-module workspaces
go.mod
declares the module path, Go version, and dependencies; go.sum
holds cryptographic hashes. Commit both. There is no venv: the toolchain resolves per-module, and builds are reproducible by construction.
Versioning is semantic import versioning: v2+
changes the import path (.../chi/v5
). Awkward at first, but it makes two major versions coexist in one build.
backend-go/
├── go.mod / go.sum
├── Makefile
├── cmd/
│ └── api/
│ ├── main.go # wiring only: config → deps → server
│ └── routes.go
├── internal/ # ← the compiler FORBIDS imports from outside this module
│ ├── handler/ # HTTP: decode, call service, encode. No business logic.
│ ├── service/ # business logic. No HTTP types, no SQL.
│ ├── repo/ # DB access (sqlx). No business rules.
│ ├── model/ # domain types shared across layers
│ └── middleware/
├── pkg/ # only for packages you intend other repos to import
├── migrations/ # goose SQL files
└── testdata/ # golden files, fixtures
internal/
is enforced by the compilerpkg/
is opt-in publicity.handler → service → repo
. If two packages need each other, extract the shared type into model/
. Go rejects import cycles at compile time, so bad layering fails the build rather than rotting.service.Agent
, not service.ServiceAgent
. The package name is part of every call site.main.go
does wiring and nothing else: read config, construct dependencies, start the server, handle SIGTERM.
gofmt -l . # formatting is not a debate; gofmt decides
go vet ./... # correctness heuristics: printf verbs, lost cancels, copied locks
go build ./...
go test -race ./...
go run ./cmd/api
go generate ./... # //go:generate directives (mocks, enums, sqlc)
govulncheck ./... # ✅ CVEs in YOUR call paths, not just in go.sum
golangci-lint
bundles the linters worth running:
linters:
enable:
- errcheck # unchecked errors ← the highest-value linter in Go
- govet
- staticcheck # the deep one: dead code, misuse, simplifications
- revive # style + doc comments
- ineffassign
- bodyclose # unclosed HTTP response bodies
- noctx # HTTP requests built without a context
- sqlclosecheck
- gosec
issues:
exclude-rules:
- path: _test\.go
linters: [gosec, errcheck]
Add air
for hot reload in development (make dev-go
in this repo), and a Makefile
so every service has the same verbs: make dev
, make test
, make lint
, make migrate-up
.
type Config struct {
DatabaseURL string
RedisURL string
Port int
APIKey string
}
func Load() (Config, error) {
c := Config{
RedisURL: "redis://localhost:6379/0", // defaults in code
Port: 8080,
}
var missing []string
for _, f := range []struct{ key string; dst *string }{
{"DATABASE_URL", &c.DatabaseURL},
{"ANTHROPIC_API_KEY", &c.APIKey},
} {
if *f.dst = os.Getenv(f.key); *f.dst == "" {
missing = append(missing, f.key)
}
}
if len(missing) > 0 {
return Config{}, fmt.Errorf("config.Load: missing env: %s", strings.Join(missing, ", "))
}
…
return c, nil
}
Validate everything in main
and exit non-zero on failure. A service that refuses to start beats one that fails on request #4000. (kelseyhightower/envconfig
or caarlos0/env
do this with struct tags if you prefer.)
Go's single static binary makes this dramatically simpler than the Python equivalent — the final image can contain only your binary.
FROM golang:1.23-bookworm AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
ARG VERSION=dev
ARG COMMIT=unknown
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
CGO_ENABLED=0 GOOS=linux go build \
-trimpath \
-ldflags="-s -w -X main.version=${VERSION} -X main.commit=${COMMIT}" \
-o /out/api ./cmd/api
FROM gcr.io/distroless/static-debian12:nonroot AS runtime
COPY --from=builder /out/api /api
COPY --from=builder /src/migrations /migrations # only if the binary applies them
USER nonroot:nonroot
EXPOSE 8080
ENV GOMEMLIMIT=450MiB GOMAXPROCS=2 # match the pod's limits (see §7.2)
ENTRYPOINT ["/api"]
Why each decision:
| Decision | Reason |
|---|---|
CGO_ENABLED=0 |
|
Removes the libc dependency, so the binary runs on scratch /distroless . If you need cgo (SQLite, some crypto), build on and ship to a matching glibc base instead. |
|
distroless/static, not alpine or ubuntu |
|
No shell, no package manager, no CVE churn from utilities you never use. Final image ≈ your binary + 2 MB. scratch is even smaller but lacks CA certs and tzdata, which any HTTPS client needs. |
|
:nonroot tag |
|
Runs as uid 65532 with no writable filesystem — satisfies runAsNonRoot policies out of the box. |
|
| Deps before source | Same caching logic as everywhere: go mod download is reused until go.sum changes. |
| BuildKit cache mounts | Keeps the module and build caches between builds without baking them into layers. |
-trimpath + -ldflags="-s -w" |
|
| Reproducible and ~25% smaller; strip only after you've decided you don't need symbols in prod profiles. | |
-X main.version=… |
|
| The binary can report its own build; invaluable when three replicas disagree. | |
No HEALTHCHECK |
|
Distroless has no shell or curl. Let Kubernetes do an HTTP probe against /healthz ; a Docker-level healthcheck would force you to ship a fatter image. |
|
GOMEMLIMIT / GOMAXPROCS |
|
| The Go runtime doesn't see cgroup limits before Go 1.25 — set them explicitly to the pod's limits (§7.1–§7.2). | |
ENTRYPOINT in exec form |
|
Your binary is PID 1 and receives SIGTERM directly — which is exactly what srv.Shutdown needs (§8.1). |
DOCKER_BUILDKIT=1 docker build \
--platform linux/amd64 \
--build-arg VERSION=1.4.2 --build-arg COMMIT=$(git rev-parse --short HEAD) \
-t agent-api:1.4.2 .
docker run --rm -p 8080:8080 --env-file .env --read-only --cap-drop=ALL agent-api:1.4.2
docker images agent-api:1.4.2 # expect ~15–30 MB total
.dockerignore
:
.git/
bin/
tmp/
*_test.go
testdata/
.env
Dockerfile
Pre-ship checklist: image under ~30 MB · docker run … --read-only
works · SIGTERM drains in-flight requests within the grace period · no secrets in docker history
· govulncheck
clean · --platform linux/amd64
when building on Apple silicon for x86 nodes.
🎯 Actionable rules
- Put everything in
internal/
unless another repo must import it.go mod tidy
,gofmt
,go vet
,golangci-lint
,govulncheck
— all in CI.- Validate config in
main
and exit non-zero on anything missing.- Ship a distroless static binary and set
GOMEMLIMIT
/GOMAXPROCS
to the pod's limits.
// .vscode/launch.json
{
"version": "0.2.0",
"configurations": [
{
"name": "Run API",
"type": "go",
"request": "launch",
"mode": "debug",
"program": "${workspaceFolder}/cmd/api",
"env": { "DATABASE_URL": "postgres://dev:dev@localhost:5432/app", "LOG_LEVEL": "debug" },
"args": ["--verbose"]
},
{
"name": "Debug current test",
"type": "go",
"request": "launch",
"mode": "test",
"program": "${fileDirname}",
"args": ["-test.run", "TestAgentUsesCalculator", "-test.v"],
"buildFlags": "-race"
},
{
"name": "Attach to container (dlv)",
"type": "go",
"request": "attach",
"mode": "remote",
"port": 2345,
"host": "127.0.0.1",
"substitutePath": [{ "from": "${workspaceFolder}", "to": "/src" }]
}
]
}
dlv exec --headless --listen=:2345 --api-version=2 --accept-multiclient /api
// .vscode/settings.json
{
"go.useLanguageServer": true,
"go.lintTool": "golangci-lint",
"go.lintOnSave": "package",
"go.testFlags": ["-race", "-count=1"],
"gopls": { "ui.semanticTokens": true, "staticcheck": true }
}
Breakpoint techniques that matter: conditional breakpoints (docID == "doc-9182"
) to catch iteration 4000 of a loop; logpoints for tracing without a rebuild; and the Goroutines panel, which is Go-specific and invaluable — it shows every live goroutine with its stack, so a leak or a deadlock is visible directly. The Debug Console evaluates expressions and lets you change variables to force an error branch.
Delve on the command line, when you're on a server:
dlv debug ./cmd/api
(dlv) break service.(*Agent).Run
(dlv) condition 1 prompt == "calculate 10 * 5"
(dlv) continue ; locals ; goroutines ; stack ; print cfg
python
import _ "net/http/pprof" // registers /debug/pprof/* on the DefaultServeMux
go func() {
// ✅ bind to localhost or an admin port — never expose pprof publicly
slog.Error("pprof", "err", http.ListenAndServe("127.0.0.1:6060", nil))
}()
go tool pprof -http=:8081 http://localhost:6060/debug/pprof/profile?seconds=30 # CPU
go tool pprof -http=:8081 http://localhost:6060/debug/pprof/heap # memory
go tool pprof http://localhost:6060/debug/pprof/allocs # all allocations
curl "http://localhost:6060/debug/pprof/goroutine?debug=2" # every goroutine's stack ← leaks
curl "http://localhost:6060/debug/pprof/block" # blocking (needs SetBlockProfileRate)
curl "http://localhost:6060/debug/pprof/mutex" # contention (needs SetMutexProfileFraction)
-http=:8081
opens an interactive flame graph in the browser. The workflow for the three problems you'll actually hit:
| Symptom | Profile | What you're looking for |
|---|---|---|
| High CPU | profile?seconds=30 |
|
| The widest frame in the flame graph | ||
| Memory grows without bound | ||
heap + goroutine?debug=2 |
||
| A goroutine count that only rises = a leak | ||
| Latency spikes at steady CPU | ||
block , mutex , gctrace=1 |
||
| Lock contention or GC pressure |
The execution tracer shows scheduling, GC, and syscalls on a timeline:
curl -o trace.out "http://localhost:6060/debug/pprof/trace?seconds=5"
go tool trace trace.out
GODEBUG=gctrace=1 ./api # one line per GC: heap, , CPU share
GODEBUG=schedtrace=1000 ./api # scheduler state every second
GODEBUG=inittrace=1 ./api # slow package init
GOTRACEBACK=all ./api # dump ALL goroutine stacks on a fatal panic
go build -gcflags='-m' ./... # escape analysis decisions
go test -race ./... # data races
go tool nm -size bin/api | sort -k2 -n | tail # what's making the binary big
kill -QUIT <pid>
on a hung Go process dumps every goroutine's stack to stderr — the Go equivalent of py-spy dump
, and it's built in.
🎯 Actionable rules
- Ship
net/http/pprof
on a private port in every service; you cannot profile what isn't instrumented.- Rising goroutine count = a leak. Check it before you check memory.
- Use the Goroutines panel /
goroutine?debug=2
for deadlocks and leaks — stacks tell you exactly who's blocked on what.GOTRACEBACK=all
andkill -QUIT
for production hangs.
What: variadic Option
functions that configure a constructor. Why: Go has no default or keyword arguments, so a growing config would otherwise mean a growing parameter list or a mutable public struct.
type Option func(*Client)
func WithTimeout(d time.Duration) Option { return func(c *Client) { c.timeout = d } }
func WithRetries(n int) Option { return func(c *Client) { c.retries = n } }
func WithLogger(l *slog.Logger) Option { return func(c *Client) { c.log = l } }
func NewClient(baseURL string, opts ...Option) (*Client, error) {
c := &Client{baseURL: baseURL, timeout: 30 * time.Second, retries: 3, log: slog.Default()}
for _, opt := range opts {
opt(c)
}
if c.baseURL == "" {
return nil, errors.New("client: baseURL is required")
}
return c, nil
}
c, err := NewClient(url, WithTimeout(90*time.Second), WithRetries(5))
Required arguments stay positional; optional ones are named and additive. Adding an option never breaks an existing caller.
http.Handler
What: func(http.Handler) http.Handler
. Why: logging, auth, tenancy, tracing, and rate limits belong around handlers, not inside them.
func Logging(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
ww := &statusWriter{ResponseWriter: w, status: http.StatusOK}
next.ServeHTTP(ww, r)
slog.Info("request",
"method", r.Method, "path", r.URL.Path,
"status", ww.status, "ms", time.Since(start).Milliseconds())
})
}
func Tenant(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t := r.Header.Get("X-Tenant")
if t == "" {
http.Error(w, "missing tenant", http.StatusUnauthorized); return
}
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), tenantKey, t)))
})
}
handler := Recoverer(RequestID(Logging(Tenant(mux)))) // or r.Use(...) with chi
The same shape works for any interface: wrap it, keep the type, add behaviour.
main
What: every dependency arrives through a constructor; main
is the only place that knows the concrete types. Why: it's Go's whole DI story — no framework, no reflection, no runtime surprises. This repo's convention ( CLAUDE.md):
init()
."
func main() {
cfg, err := config.Load()
if err != nil { fatal(err) }
db, err := sqlx.Connect("pgx", cfg.DatabaseURL)
if err != nil { fatal(err) }
defer db.Close()
var (
docs = repo.NewDocs(db) // concrete
llm = llmclient.New(cfg.APIKey) // concrete
svc = service.NewAgent(docs, llm) // takes interfaces
h = handler.New(svc) // takes an interface
)
…
}
Read main
top to bottom and you know the entire architecture. Every layer is testable because every layer takes interfaces it doesn't construct.
What: bounded parallel map
, order preserved. Why: you'll write this loop in every AI service — embed, rerank, enrich, fan out to tools.
// ParallelMap applies f to every element with at most n concurrent calls.
// Results keep the input order; the first error cancels the rest.
func ParallelMap[T, U any](ctx context.Context, in []T, n int, f func(context.Context, T) (U, error)) ([]U, error) {
out := make([]U, len(in))
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(n)
for i, v := range in {
g.Go(func() error {
u, err := f(ctx, v)
if err != nil {
return fmt.Errorf("item %d: %w", i, err)
}
out[i] = u // distinct index per goroutine → no lock needed
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
return out, nil
}
vecs, err := ParallelMap(ctx, chunks, 8, embedOne)
What: start dependencies, block on a signal, shut down in reverse. Why: rolling deploys happen constantly; dropping in-flight streams on every deploy is a self-inflicted SLO breach.
func run(ctx context.Context, cfg config.Config) error {
ctx, stop := signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM)
defer stop()
srv := newServer(cfg)
errc := make(chan error, 1)
go func() { errc <- srv.ListenAndServe() }()
select {
case err := <-errc:
if !errors.Is(err, http.ErrServerClosed) { return err }
case <-ctx.Done():
slog.Info("shutting down")
}
shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return srv.Shutdown(shutdownCtx) // stop accepting, drain in-flight
}
func main() {
if err := run(context.Background(), cfg); err != nil {
slog.Error("fatal", "err", err); os.Exit(1)
}
}
Putting the body in run(ctx) error
— with main
only handling the exit code — makes the whole startup path testable.
What: type TenantID string
, type Role string
. Why: the compiler stops you passing a user ID where a tenant ID belongs, at zero runtime cost.
type (
TenantID string
DocID string
)
func (r *Repo) Search(ctx context.Context, t TenantID, q string) ([]Doc, error) { … }
r.Search(ctx, TenantID(hdr), q) // ✅ explicit conversion at the boundary
r.Search(ctx, userID, q) // ❌ compile error — exactly what you want
func (a *Agent) Run(ctx context.Context, userInput string) (AgentResponse, error) {
a.AddMessage(RoleUser, userInput)
var reply string
switch lower := strings.ToLower(userInput); {
case strings.Contains(lower, "calculate"):
expr := strings.TrimSpace(strings.SplitN(lower, "calculate", 2)[1])
res, err := a.tools.Dispatch(ctx, "calculator", mustArgs("expression", expr))
if err != nil {
return AgentResponse{}, fmt.Errorf("agent.Run: %w", err)
}
a.results = append(a.results, res)
reply = "Result: " + res.Output
case strings.Contains(lower, "count"):
res, err := a.tools.Dispatch(ctx, "word_count", mustArgs("text", userInput))
if err != nil {
return AgentResponse{}, fmt.Errorf("agent.Run: %w", err)
}
a.results = append(a.results, res)
top := make([]string, 0, 3)
for _, kv := range topN(res.Counts, 3) {
top = append(top, fmt.Sprintf("%s=%d", kv.Key, kv.Count))
}
reply = "Top words: " + strings.Join(top, ", ")
default:
reply = fmt.Sprintf("Echo [%s]: %s", a.cfg.Name, userInput)
}
a.AddMessage(RoleAssistant, reply)
return AgentResponse{
Messages: a.History(),
ToolResults: a.results,
TotalSteps: 1,
Status: StatusOK,
}, nil
}
Everything in one method: ctx
first, typed Role
constants, switch
with an init statement, error wrapping at every boundary, preallocated slices, and a struct return instead of a tuple.
🎯 Actionable rules
- Functional options for anything with more than two optional settings.
- Wire concrete types in
main
; pass interfaces everywhere else.- Middleware for cross-cutting concerns;
defer
for resources.- Named domain types for identifiers — free compile-time safety.
Twenty-two rewrites you can apply in your next code review.
// ❌ the failure vanishes; the zero value flows onward
data, _ := json.Marshal(payload)
// ✅ handle it, or say in writing why it can't happen
data, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("handler.Query: marshal response: %w", err)
}
// ❌ "sql: no rows in result set" — from where? which id? which layer?
if err != nil { return err }
// ✅ the chain reads like a stack trace you designed
if err != nil { return fmt.Errorf("repo.GetDoc(%s): %w", id, err) }
// ❌ the success case is buried three levels deep
if resp != nil {
if resp.StatusCode == 200 {
if body, err := io.ReadAll(resp.Body); err == nil {
return parse(body)
}
}
}
return nil, errors.New("failed")
// ✅ fail fast, one indent level, every error distinguishable
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("llm.Complete: status %d", resp.StatusCode)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, maxBody))
if err != nil {
return nil, fmt.Errorf("llm.Complete: read body: %w", err)
}
return parse(body)
// ❌ leaks a connection on every error path — and exhausts the pool under load
resp, err := client.Do(req)
if err != nil { return err }
body, err := io.ReadAll(resp.Body)
resp.Body.Close()
// ✅
resp, err := client.Do(req)
if err != nil { return fmt.Errorf("fetch: %w", err) }
defer resp.Body.Close()
// ❌ new pool per call, and no timeout: a hung upstream hangs you forever
resp, err := (&http.Client{}).Get(url)
js
// ✅ package-level, pooled, bounded (see §8.2)
var client = &http.Client{Timeout: 60 * time.Second, Transport: tunedTransport}
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
resp, err := client.Do(req)
js
// ❌ ~log(n) reallocations and copies
var out []Doc
for _, id := range ids { out = append(out, load(id)) }
// ✅ one allocation
out := make([]Doc, 0, len(ids))
for _, id := range ids { out = append(out, load(id)) }
// ❌ O(n²): every += copies the whole prompt
prompt := ""
for _, m := range history { prompt += m.Role + ": " + m.Content + "\n" }
js
// ✅ O(n)
var b strings.Builder
for _, m := range history {
fmt.Fprintf(&b, "%s: %s\n", m.Role, m.Content)
}
prompt := b.String()
// ❌ 10 000 goroutines, 10 000 sockets, instant 429s, no error handling
for _, id := range ids {
go fetch(id)
}
// ✅ at most 8 in flight, first error cancels the rest
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(8)
for _, id := range ids {
g.Go(func() error { return fetch(ctx, id) })
}
if err := g.Wait(); err != nil { return fmt.Errorf("service.LoadAll: %w", err) }
// ❌ if nobody ever receives, this goroutine (and its captures) leaks forever
go func() { results <- expensive() }()
// ✅ cancellation always wins
go func() {
select {
case results <- expensive():
case <-ctx.Done():
}
}()
// ❌ the client hung up 20 seconds ago; you're still paying for tokens
func (s *Service) Answer(q string) (string, error) {
return s.llm.Complete(context.Background(), q)
}
// ✅ ctx first, always — cancellation flows all the way down
func (s *Service) Answer(ctx context.Context, q string) (string, error) {
return s.llm.Complete(ctx, q)
}
// ❌ a missing tool and a tool with score 0 are indistinguishable
if scores[name] == 0 { skip() }
// ✅
score, ok := scores[name]
if !ok { return fmt.Errorf("unknown tool %q", name) }
// ❌ a 9-method interface exported by the implementer — impossible to fake in a test
package llm
type Provider interface {
Complete(...); Stream(...); Embed(...); Tokenize(...); Models(...); /* … */
}
// ✅ each consumer declares the one or two methods it needs
package service
type Completer interface {
Complete(ctx context.Context, prompt string) (string, error)
}
// ❌ `go vet` error: passes a copy of the lock; the copy protects nothing
func (c Cache) Get(k string) string { c.mu.RLock(); … }
// ✅ pointer receiver, consistently across all methods
func (c *Cache) Get(k string) string { c.mu.RLock(); defer c.mu.RUnlock(); … }
// ❌ "fatal error: concurrent map writes" — unrecoverable, takes the process down
var cache = map[string][]float32{}
go func() { cache[k] = v }()
// ✅ mutex next to the data it protects (or a channel-owned goroutine)
type Cache struct {
mu sync.RWMutex
m map[string][]float32
}
errors.Is
, not ==
// ❌ breaks the moment any layer wraps the error
if err == sql.ErrNoRows { return NotFound }
// ✅ traverses the whole wrap chain
if errors.Is(err, sql.ErrNoRows) { return NotFound }
// ❌ err != nil is TRUE even when everything succeeded
func do() error {
var e *ToolError // nil pointer…
return e // …wrapped in a non-nil interface
}
js
// ✅ return the literal nil
func do() error {
var e *ToolError
if failed { e = &ToolError{…}; return e }
return nil
}
// ❌ keeps the entire 50 MB document alive for a 100-byte snippet
snippet := bigDoc[:100]
cache.Put(id, snippet)
// ✅ copy out what you keep
cache.Put(id, slices.Clone(bigDoc[:100]))
// ❌ buffers the whole body, silently ignores typo'd client fields
b, _ := io.ReadAll(r.Body)
json.Unmarshal(b, &in)
// ✅ streaming, bounded, strict
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20))
dec.DisallowUnknownFields()
if err := dec.Decode(&in); err != nil {
http.Error(w, "invalid body", http.StatusBadRequest); return
}
map[string]any
// ❌ every number becomes float64; every access is an unchecked assertion
var m map[string]any
json.Unmarshal(b, &m)
n := int(m["max_tokens"].(float64)) // panics on any surprise
js
// ✅ the shape is documented, validated, and autocompleted
var in struct {
MaxTokens int `json:"max_tokens"`
Model string `json:"model"`
}
// ❌ one malformed model response kills the process
func mustParse(b []byte) Call {
var c Call
if err := json.Unmarshal(b, &c); err != nil { panic(err) }
return c
}
// ✅ expected failures are values
func parseCall(b []byte) (Call, error) {
var c Call
if err := json.Unmarshal(b, &c); err != nil {
return Call{}, fmt.Errorf("agent.parseCall: %w", err)
}
return c, nil
}
// ❌ the same failure appears five times in the logs, at five layers
if err != nil {
slog.Error("query failed", "err", err)
return err
}
// ✅ lower layers add context; only the handler logs
if err != nil { return fmt.Errorf("repo.GetDoc: %w", err) } // repo
…
if err != nil { // handler
slog.Error("query failed", "err", err, "path", r.URL.Path)
http.Error(w, "internal error", http.StatusInternalServerError)
}
// ❌ what is 30? seconds? retries? tokens?
ctx, cancel := context.WithTimeout(ctx, 30) // 30 NANOSECONDS — instant timeout
// ✅ typed durations and named constants make the unit impossible to get wrong
const toolTimeout = 30 * time.Second
ctx, cancel := context.WithTimeout(ctx, toolTimeout)
defer cancel()
| Belief | Reality |
|---|---|
| "Goroutines are free" | Cheap, not free. Unbounded goroutines = unbounded memory, sockets, and downstream load. |
| "Channels are the answer to everything" | A mutex around a map is simpler and faster. Channels are for transferring ownership, not for protecting state. |
| "Buffered channels prevent blocking" | They delay it. A full buffer blocks exactly like an unbuffered one — the buffer just hides the backpressure until it's worse. |
"close(ch) stops the consumer" |
|
| It signals no more values. To stop work, cancel the context. | |
| "Go has no memory leaks; there's a GC" | Goroutine leaks, retained slice backing arrays, and unstopped tickers are all leaks the GC can't help with. |
"err != nil everywhere is boilerplate" |
|
| It's the feature. Every failure path is visible and testable — the reason Go services behave predictably. | |
| "Empty interface = Python's dynamic typing" | |
any costs you every compile-time guarantee, plus an allocation. Use concrete types. |
|
| "Go is slow at JSON / it needs a framework" | |
encoding/json handles most loads; net/http is a production HTTP/2 server. Reach for libraries after profiling. |
|
"sync.Map is a faster map" |
|
| It's slower for most workloads. It exists for two specific access patterns. | |
| "GOMAXPROCS handles containers" | Only from Go 1.25. Before that, set it from the cgroup quota or your 500m pod spawns dozens of Ps. |
| "The GC keeps me under the memory limit" | Not without GOMEMLIMIT . Otherwise the heap grows past the cgroup limit and the OOM killer wins. |
| "Generics replace interfaces" | Different tools. Interfaces for polymorphism, generics for eliminating duplicate code over types. |
| "A panic in a goroutine is caught by my middleware" | |
recover is per-goroutine. An unrecovered panic anywhere kills the process. |
|
| "Interfaces should be defined next to the implementation" | Java habit. In Go the consumer declares what it needs. |
1. interface{}/any in your own APIs. It pushes type errors to runtime and allocates. → Concrete types, or generics if you truly need several.
2. Package utils/common/helpers. It becomes a dependency magnet and an import-cycle factory. → Name packages for what they provide:
tokens
, retry
, chunk
.3. Stuttering names. service.ServiceAgent
, model.ModelMessage
. → The package qualifies the name: service.Agent
.
4. Storing context.Context in a struct. It outlives the request and cancellation stops matching reality. → Pass it as the first parameter, every time.
5. Global mutable state. var db *sql.DB
at package scope makes tests order-dependent and races invisible. → Constructor injection (§13.3).
6. Giant interfaces / interfaces with one implementation. Premature abstraction with a compile-time cost. → Write the concrete type; extract an interface at the consumer when a second implementation (or a test fake) appears.
7. defer inside a loop. Resources accumulate until the function returns (§3.4). → Wrap the body in a function.
8. Ignoring rows.Err() / scanner.Err().
for rows.Next()
ending doesn't mean success — it may have failed mid-iteration. → Check the error after the loop, always.9. Unbounded append on request data. An unbounded history slice or in-memory result buffer is an OOM on a slow day. → Cap it: window the history, stream the results.
10. Time-based tests. time.Sleep(100*time.Millisecond)
to "wait for the goroutine" is flaky by construction. → Synchronize with a channel or WaitGroup
; inject a clock.
11. Reinventing errgroup, singleflight, or rate. These are hard to get right and already exist in
golang.org/x/...
.12. log.Fatal outside main. It calls
os.Exit
, skipping every defer
— no flush, no shutdown, no cleanup. → Return an error; let main
decide.13. Struct literals without field names. AgentConfig{"a", "b", 0.7}
silently breaks when a field is inserted. → Always Field: value
.
14. Exporting everything. Every exported identifier is API you must keep working. → Start lowercase; export on demand.
15. Swallowing ctx.Err(). Treating cancellation as a generic failure produces 500s for clients that simply disconnected. → Check
errors.Is(err, context.Canceled)
and return early without logging noise.| Days | Focus | Ship this |
|---|---|---|
| 1–3 | §1–§2: syntax, slices, maps, structs | A CLI that chunks a file and prints word stats |
| 4–6 |
§3–§4: functions, defer , methods, interfaces |
A Tool interface with two implementations and a registry |
| 7–9 |
§5: errors, wrapping, Is /As
|
A typed error hierarchy with sentinels and errors.As handling |
| 10–14 | §6: goroutines, channels, context | A bounded worker pool that embeds 10k chunks and cancels cleanly |
| 15–17 |
§8: net/http , json , slog
|
A JSON API with timeouts, middleware, and graceful shutdown |
| 18–20 | §9: streaming, retries, limits | An SSE endpoint proxying a real model with backpressure |
| 21–23 | §10: tests, fakes, benchmarks | Table-driven tests + httptest + a fuzz target, all -race clean |
| 24–26 | §7, §12: runtime and profiling | Profile it, cut allocations 50%, write down what you learned |
| 27–30 | §11, §13–§15 | Distroless image, CI with lint+race, refactor against §14 |
// Declarations
x := 5 // infer var x int // zero value 0
m := make(map[string]int, 100) // ✅ never a nil map you write to
s := make([]T, 0, n) // preallocate
v, ok := m[k] // comma-ok
s = append(s, xs...) // always reassign
// Errors
if err != nil { return fmt.Errorf("pkg.Func: %w", err) }
errors.Is(err, ErrNotFound) · errors.As(err, &myErr) · errors.Join(errs...)
defer resp.Body.Close() // right after the error check
// Concurrency
g, ctx := errgroup.WithContext(ctx); g.SetLimit(8); g.Go(func() error { … }); g.Wait()
select { case v := <-ch: … case <-ctx.Done(): return ctx.Err() }
var mu sync.RWMutex; mu.RLock(); defer mu.RUnlock()
ctx, cancel := context.WithTimeout(ctx, 30*time.Second); defer cancel()
// Interfaces
type Completer interface { Complete(context.Context, string) (string, error) }
var _ Completer = (*Client)(nil) // compile-time check
switch x := v.(type) { case string: … }
// Format
%v %+v %#v %q %T %w %.2f %d
// Commands
go test -race ./... · go test -bench=. -benchmem · go test -fuzz=Fuzz
go vet ./... · golangci-lint run · govulncheck ./... · go mod tidy
go tool pprof -http=:8081 http://localhost:6060/debug/pprof/profile?seconds=30
curl localhost:6060/debug/pprof/goroutine?debug=2 # leak hunting
GODEBUG=gctrace=1 ./api · GOMEMLIMIT=450MiB · kill -QUIT <pid>
pkg.Func:
context; log once at the top.select
has <-ctx.Done()
.ctx
is the first parametermain
init()
.-race
in CI, pprof
in production.GOMEMLIMIT
and GOMAXPROCS
Where to go next:[🐍 Python for AI Developers]for the other half of the stack,[🏗️ Building High-Quality AI Agents]for the architecture on top, and[🏢 Enterprise-Ready AI Agents]for multi-tenancy, security, and scale.
Go gives you fewer ways to write it, so there are fewer ways to get it wrong. Learn error, interface, defer, context, and the scheduler — the rest of the language fits on one page, which was always the point.
If you found this helpful, let me know by leaving a 👍 or a comment!, or if you think this post could help someone, feel free to share it! Thank you very much! 😃