{"slug": "inferno-kubernetes-native-ai-inference-operator-10-day-expert-learning-plan", "title": "INFERNO: Kubernetes-Native AI Inference Operator - 10-Day Expert Learning Plan", "summary": "A developer outlines a 10-day expert learning plan for building a Kubernetes-native AI inference operator using Kubebuilder. The plan covers required Go and Kubernetes knowledge, the problem of boilerplate in operator development, and the workflow of generating CRDs, controllers, and deployment manifests.", "body_md": "**Project:** Kubernetes-Native AI Inference Operator\n\n**Duration:** 10 days\n\n**Level:** Expert\n\n**Goal:** Build a working Kubernetes controller that manages AI inference workloads\n\n**Go Knowledge Required:**\n\n- Basic syntax (variables, functions, structs)\n- Interfaces\n- Error handling with\n`error`\n\ntype - Goroutines basics\n- Package management with\n`go mod`\n\n**If you're weak on these**, take 1 day to review:\n\n```\n# Quick Go refresher resources\n- https://gobyexample.com/ (30 min)\n- Read: https://tour.golang.org/methods/1 (interfaces, 20 min)\n- Practice: Write a simple HTTP server\n```\n\n**Kubernetes Knowledge Required:**\n\n- What Pods, Deployments, Services are\n- How to use kubectl (get, apply, describe, logs)\n- YAML basics\n- What a namespace is\n\n**If weak, do:**\n\n```\n# Quick k8s refresher\n- kubectl cheatsheet: https://kubernetes.io/docs/reference/kubectl/cheatsheet/\n- Deploy a simple app: kubectl apply -f nginx.yaml\n- Check it: kubectl get pods, kubectl logs\n- Clean up: kubectl delete -f nginx.yaml\n```\n\n**The Problem It Solves:**\n\nBuilding Kubernetes operators is hard. You need:\n\n- API groups, versions, kinds\n- CRD manifests\n- Reconciliation loops\n- RBAC rules\n- Deployment manifests\n- Testing setup\n\nAll of this is **boilerplate**. Kubebuilder is a **scaffolding tool** that generates all this for you.\n\n**What Kubebuilder Does:**\n\n```\nkubebuilder init          # Create project skeleton\nkubebuilder create api    # Generate API + controller template\nmake manifests            # Generate CRD YAML\nmake install              # Install CRD into cluster\nmake run                  # Run controller locally\nmake deploy               # Deploy controller to cluster\n```\n\n**What Kubebuilder Generates:**\n\n```\nmy-operator/\n├── api/v1alpha1/               # Your API definitions (the YAML structure)\n│   ├── myresource_types.go      # Go structs that become YAML\n│   └── myresource_webhook.go    # Validation webhooks (advanced)\n├── controllers/                 # Your business logic\n│   └── myresource_controller.go # The reconciliation loop\n├── config/\n│   ├── crd/                     # Kubernetes CRD manifests (generated)\n│   ├── manager/                 # Operator deployment YAML (generated)\n│   ├── rbac/                    # RBAC rules (generated)\n│   └── samples/                 # Example CRs you create\n├── main.go                      # Operator entry point\n├── Dockerfile                   # Container image definition\n├── Makefile                     # Build commands\n└── go.mod                       # Go dependencies\n```\n\n**The Workflow:**\n\n```\nYou Write:                          Kubebuilder Generates:\n┌──────────────────────────┐       ┌──────────────────────────┐\n│ 1. API types (Go structs)│ ────→ │ 1. CRD YAML manifests   │\n│ 2. Controller logic      │       │ 2. RBAC rules           │\n│ 3. Reconcile function    │       │ 3. OpenAPI docs         │\n│ 4. Tests                 │       │ 4. Deployment manifest  │\n└──────────────────────────┘       └──────────────────────────┘\n```\n\n**Key Files You'll Edit:**\n\n— Define what users can write in YAML`api/v1alpha1/myresource_types.go`\n\n— Write the reconciliation logic`controllers/myresource_controller.go`\n\n— Example YAML for testing`config/samples/*.yaml`\n\n**Everything else is generated or boilerplate.**\n\nThis is the **heart** of every Kubernetes operator:\n\n```\n┌─────────────────────────────────────────────┐\n│  Kubernetes API Server                      │\n│  (stores all resources)                     │\n└────────────────┬────────────────────────────┘\n                 │\n                 │ Watch: \"Tell me when anything changes\"\n                 ↓\n┌─────────────────────────────────────────────┐\n│  Controller (your code)                     │\n│                                             │\n│  func (r *MyReconciler) Reconcile(...) {   │\n│    1. Read the current CR                  │\n│    2. Check what's needed                  │\n│    3. Create/update Deployments, Services  │\n│    4. Update the CR's status               │\n│    5. Return (no change) or Requeue        │\n│  }                                         │\n└────────────────┬────────────────────────────┘\n                 │\n                 │ Create/Update Resources\n                 ↓\n┌─────────────────────────────────────────────┐\n│  Kubernetes Cluster                         │\n│  (Deployments, Services, etc.)              │\n└─────────────────────────────────────────────┘\n```\n\n**When does Reconcile() get called?**\n\n- Someone creates/updates/deletes an InferenceService\n- Kubernetes notifies the controller: \"Something changed!\"\n- Controller calls\n`Reconcile()`\n\n- Controller reads the CR and creates/updates resources\n\n**What should Reconcile() do?**\n\n```\nfunc (r *MyReconciler) Reconcile(ctx, req) (Result, error) {\n    // 1. Read the resource\n    cr := &MyResource{}\n    r.Get(ctx, req.NamespacedName, cr)\n    \n    // 2. Check: Does desired state == actual state?\n    deployment := &appsv1.Deployment{}\n    err := r.Get(ctx, deploymentName, deployment)\n    \n    if err != nil {\n        // 3a. Desired state missing: CREATE it\n        newDeployment := r.constructDeployment(cr)\n        r.Create(ctx, newDeployment)\n    } else {\n        // 3b. Desired state exists: UPDATE it\n        r.Update(ctx, deployment)\n    }\n    \n    // 4. Update status\n    cr.Status.Ready = true\n    r.Status().Update(ctx, cr)\n    \n    // 5. Tell k8s \"I'm done, check again later\"\n    return Result{RequeueAfter: 10*time.Second}, nil\n}\n```\n\n**The Goal:** Make actual state match desired state. Forever.\n\nThis plan assumes you have **basic** Go and Kubernetes knowledge. If not, take 1-2 days to refresh using the links above. You'll build INFERNO in phases:\n\n**Days 1-3**: Kubernetes controller fundamentals & Kubebuilder setup** Days 4-6**: Core operator logic & InferenceService CRD** Days 7-9**: Reconciliation, Deployment management, and testing** Day 10**: Polish, integration testing, and documentation\n\nEach day has theory + hands-on code. By Day 10, you'll have a working operator that can deploy, scale, and manage fake inference workloads.\n\n**Struct tags**— How Go metadata works (`json:\"field\"`\n\n,`kubebuilder:validation:`\n\n)**Interfaces**— The`client.Client`\n\ninterface (implements Get, Create, Update)**Goroutines & channels**— Controller-runtime uses these under the hood** Error handling**— Pattern of`if err != nil { return err }`\n\n**Package management**— Using`import`\n\nfor external libraries\n\n- Understand how Kubernetes controllers work (watch → reconcile → act)\n- Learn the controller-runtime library\n- Set up your first Kubebuilder project\n- Understand CRDs (Custom Resource Definitions)\n\n**Kubernetes Controller Pattern:**\n\n```\n┌─────────────────────────────────┐\n│  Watch: Is desired != actual?   │\n│  If yes → Reconcile             │\n│  Update cluster state           │\n│  Requeue if needed              │\n└─────────────────────────────────┘\n```\n\n**controller-runtime**: The library that manages this loop** Reconciliation**: The core function that makes changes** Leader election**: How operators stay safe in HA setups** Finalizers**: Cleanup when resources are deleted\n\n```\n# Create project structure\ngo mod init github.com/yourusername/inferno\nkubebuilder init --domain inferno.io --repo github.com/yourusername/inferno\n\n# Create the InferenceService API\nkubebuilder create api --group inference --version v1alpha1 --kind InferenceService\n```\n\n`api/v1alpha1/inferenceservice_types.go`\n\n— Your CRD schema`controllers/inferenceservice_controller.go`\n\n— Your reconciliation logic`config/crd/`\n\n— Kubernetes CRD manifests`config/manager/`\n\n— Operator deployment manifests\n\n- Project initialized with kubebuilder\n- InferenceService CRD created (empty, will fill tomorrow)\n- Controller file exists and compiles\n- Understand the reconciliation loop by reading generated code\n\n[Kubebuilder Book](https://book.kubebuilder.io/)— Read chapters 1-3[controller-runtime Overview](https://pkg.go.dev/sigs.k8s.io/controller-runtime)- Watch: \"How Kubernetes Controllers Work\" (10 min explainer on YouTube)\n\n**Struct field tags**— How`json:\"field\"`\n\ncontrols YAML marshalling**Kubebuilder validation tags**—`+kubebuilder:validation:Required`\n\n,`Enum=`\n\n,`Min=`\n\n**Pointer vs value types**— When to use`*int32`\n\nvs`int`\n\n**Slice operations**— Working with arrays (`[]TypeName`\n\n)**Type embedding**— How`metav1.TypeMeta`\n\nand`metav1.ObjectMeta`\n\ninherit fields\n\n- Design your CRD schema (the YAML users will write)\n- Understand Go struct tags for validation & OpenAPI\n- Implement status subresource\n- Write validation rules\n\nThe user wants to write:\n\n```\napiVersion: inference.inferno.io/v1alpha1\nkind: InferenceService\nmetadata:\n  name: llama\nspec:\n  model:\n    uri: s3://models/llama-3\n    framework: vllm\n    version: \"1\"\n  replicas:\n    min: 1\n    max: 5\n  resources:\n    requests:\n      cpu: \"4\"\n      memory: \"16Gi\"\n      gpu: \"1\"\n  autoscaling:\n    enabled: true\n    targetConcurrency: 10\n  traffic:\n    - version: \"1\"\n      weight: 100\n```\n\nYou need to translate this into Go structs.\n\nEdit `api/v1alpha1/inferenceservice_types.go`\n\n:\n\n```\npackage v1alpha1\n\nimport (\n\tcorev1 \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// ModelSpec defines the model to deploy\ntype ModelSpec struct {\n\t// URI is the location of the model (s3://, http://, local path)\n\t// +kubebuilder:validation:Required\n\tURI string `json:\"uri\"`\n\n\t// Framework is the inference runtime (vllm, triton, etc)\n\t// +kubebuilder:validation:Enum=vllm;triton;fake\n\tFramework string `json:\"framework\"`\n\n\t// Version is the model version tag\n\t// +kubebuilder:validation:Required\n\tVersion string `json:\"version\"`\n}\n\n// ResourceRequirements mirrors k8s resource requests\ntype ResourceRequirements struct {\n\tCPU    string `json:\"cpu,omitempty\"`\n\tMemory string `json:\"memory,omitempty\"`\n\tGPU    string `json:\"gpu,omitempty\"`\n}\n\n// AutoscalingSpec defines HPA behavior\ntype AutoscalingSpec struct {\n\tEnabled           bool `json:\"enabled\"`\n\tTargetConcurrency int  `json:\"targetConcurrency,omitempty\"`\n\tMinReplicas       int  `json:\"minReplicas,omitempty\"`\n\tMaxReplicas       int  `json:\"maxReplicas,omitempty\"`\n}\n\n// TrafficWeight defines canary traffic split\ntype TrafficWeight struct {\n\tVersion string `json:\"version\"`\n\tWeight  int    `json:\"weight\"` // 0-100\n}\n\n// InferenceServiceSpec defines the desired state\ntype InferenceServiceSpec struct {\n\tModel       ModelSpec             `json:\"model\"`\n\tReplicas    ReplicaConfig         `json:\"replicas\"`\n\tResources   ResourceRequirements  `json:\"resources,omitempty\"`\n\tAutoscaling AutoscalingSpec       `json:\"autoscaling,omitempty\"`\n\tTraffic     []TrafficWeight       `json:\"traffic,omitempty\"`\n}\n\n// ReplicaConfig defines scaling bounds\ntype ReplicaConfig struct {\n\tMin int `json:\"min\"`\n\tMax int `json:\"max\"`\n}\n\n// InferenceServiceStatus defines the observed state\ntype InferenceServiceStatus struct {\n\t// Phase: Pending, Running, Failed\n\tPhase string `json:\"phase,omitempty\"`\n\n\t// ReadyReplicas: how many pods are ready\n\tReadyReplicas int `json:\"readyReplicas\"`\n\n\t// Message for debugging\n\tMessage string `json:\"message,omitempty\"`\n\n\t// LastUpdateTime when the controller last reconciled\n\tLastUpdateTime *metav1.Time `json:\"lastUpdateTime,omitempty\"`\n\n\t// Conditions for events\n\tConditions []metav1.Condition `json:\"conditions,omitempty\"`\n}\n\n// +kubebuilder:object:root=true\n// +kubebuilder:subresource:status\n// +kubebuilder:resource:shortName=isvc;scope=Namespaced\ntype InferenceService struct {\n\tmetav1.TypeMeta   `json:\",inline\"`\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\tSpec   InferenceServiceSpec   `json:\"spec,omitempty\"`\n\tStatus InferenceServiceStatus `json:\"status,omitempty\"`\n}\n\n// +kubebuilder:object:root=true\ntype InferenceServiceList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\tItems           []InferenceService `json:\"items\"`\n}\n```\n\n: Generate validation, OpenAPI docs, RBAC`+kubebuilder:`\n\nannotations**Status subresource**: Separate spec (desired) from status (actual)** Validation tags**:`Required`\n\n,`Enum`\n\n,`Min`\n\n,`Max`\n\n— enforced at API server**Conditions**: Standard k8s pattern for tracking async operations\n\n- Complete\n`inferenceservice_types.go`\n\nwith all structs - Run\n`make generate`\n\nto create CRD manifests - CRD installs without errors:\n`kubectl apply -f config/crd/`\n\n- Can create a test InferenceService YAML\n\n[Kubebuilder CRD Tutorial](https://book.kubebuilder.io/cronjob-tutorial/api-design.html)[Go API Conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md)- Kubernetes API reference for\n`metav1.Condition`\n\n**Context**— How`context.Context`\n\ncontrols cancellation and timeouts**The**—`client`\n\ninterface`Get()`\n\n,`Create()`\n\n,`Update()`\n\nmethods**Error wrapping**—`if err != nil`\n\npatterns**Named return types**—`(ctrl.Result, error)`\n\nconvention**Defer statements**— Cleanup code (not used yet, but will on Day 7)** Type assertions**— Checking error types (`apierrors.IsNotFound(err)`\n\n)\n\n- Understand the reconciliation function signature\n- Learn how to watch resources and trigger reconciliation\n- Implement basic reconciliation logic\n- Set up Deployment creation from InferenceService\n\nThe controller's main job:\n\n**Watch** InferenceService CRs**Reconcile**: For each CR, ensure a Deployment exists with the right spec** Update status**with what it found** Requeue**if needed (e.g., \"check again in 10s\")\n\nEdit `controllers/inferenceservice_controller.go`\n\n:\n\n```\npackage controllers\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\tappsv1 \"k8s.io/api/apps/v1\"\n\tcorev1 \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\t\"sigs.k8s.io/controller-runtime/pkg/log\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\tinferencev1alpha1 \"github.com/yourusername/inferno/api/v1alpha1\"\n)\n\n// InferenceServiceReconciler reconciles an InferenceService object\ntype InferenceServiceReconciler struct {\n\tclient.Client\n\tScheme *runtime.Scheme\n}\n\n// +kubebuilder:rbac:groups=inference.inferno.io,resources=inferenceservices,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=inference.inferno.io,resources=inferenceservices/status,verbs=get;update;patch\n// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=\"\",resources=services,verbs=get;list;watch;create;update;patch;delete\n\nfunc (r *InferenceServiceReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := log.FromContext(ctx)\n\n\t// Step 1: Fetch the InferenceService\n\tvar inferenceService inferencev1alpha1.InferenceService\n\tif err := r.Get(ctx, req.NamespacedName, &inferenceService); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Info(\"InferenceService not found, ignoring\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.Error(err, \"Failed to fetch InferenceService\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tlog.Info(\"Reconciling InferenceService\", \"name\", inferenceService.Name)\n\n\t// Step 2: Create or update Deployment\n\tdeployment := &appsv1.Deployment{}\n\tdeploymentName := types.NamespacedName{\n\t\tName:      inferenceService.Name,\n\t\tNamespace: inferenceService.Namespace,\n\t}\n\n\tif err := r.Get(ctx, deploymentName, deployment); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\t// Create new Deployment\n\t\t\tdeployment = r.constructDeployment(&inferenceService)\n\t\t\tif err := controllerutil.SetControllerReference(&inferenceService, deployment, r.Scheme); err != nil {\n\t\t\t\tlog.Error(err, \"Failed to set controller reference\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\t\t\tif err := r.Create(ctx, deployment); err != nil {\n\t\t\t\tlog.Error(err, \"Failed to create Deployment\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\t\t\tlog.Info(\"Created Deployment\", \"deployment\", deployment.Name)\n\t\t} else {\n\t\t\tlog.Error(err, \"Failed to fetch Deployment\")\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t} else {\n\t\t// Update existing Deployment\n\t\tdeployment = r.constructDeployment(&inferenceService)\n\t\tif err := r.Update(ctx, deployment); err != nil {\n\t\t\tlog.Error(err, \"Failed to update Deployment\")\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t\tlog.Info(\"Updated Deployment\", \"deployment\", deployment.Name)\n\t}\n\n\t// Step 3: Update status\n\tinferenceService.Status.Phase = \"Running\"\n\tinferenceService.Status.ReadyReplicas = int(*deployment.Spec.Replicas)\n\tinferenceService.Status.LastUpdateTime = &metav1.Time{Time: time.Now()}\n\n\tif err := r.Status().Update(ctx, &inferenceService); err != nil {\n\t\tlog.Error(err, \"Failed to update status\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tlog.Info(\"Reconciliation complete\")\n\treturn ctrl.Result{}, nil\n}\n\n// constructDeployment builds the Deployment spec from InferenceService\nfunc (r *InferenceServiceReconciler) constructDeployment(isvc *inferencev1alpha1.InferenceService) *appsv1.Deployment {\n\treplicas := int32(isvc.Spec.Replicas.Min)\n\n\tlabels := map[string]string{\n\t\t\"app\":      isvc.Name,\n\t\t\"operator\": \"inferno\",\n\t}\n\n\tdeployment := &appsv1.Deployment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName:      isvc.Name,\n\t\t\tNamespace: isvc.Namespace,\n\t\t},\n\t\tSpec: appsv1.DeploymentSpec{\n\t\t\tReplicas: &replicas,\n\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\tMatchLabels: labels,\n\t\t\t},\n\t\t\tTemplate: corev1.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: labels,\n\t\t\t\t},\n\t\t\t\tSpec: corev1.PodSpec{\n\t\t\t\t\tContainers: []corev1.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName:  \"inference-server\",\n\t\t\t\t\t\t\tImage: \"inference-server:fake\", // TODO: vLLM later\n\t\t\t\t\t\t\tPorts: []corev1.ContainerPort{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tContainerPort: 8000,\n\t\t\t\t\t\t\t\t\tName:          \"http\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tEnv: []corev1.EnvVar{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName:  \"MODEL_URI\",\n\t\t\t\t\t\t\t\t\tValue: isvc.Spec.Model.URI,\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName:  \"MODEL_VERSION\",\n\t\t\t\t\t\t\t\t\tValue: isvc.Spec.Model.Version,\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tResources: corev1.ResourceRequirements{\n\t\t\t\t\t\t\t\tRequests: corev1.ResourceList{\n\t\t\t\t\t\t\t\t\tcorev1.ResourceCPU:    resource.MustParse(isvc.Spec.Resources.CPU),\n\t\t\t\t\t\t\t\t\tcorev1.ResourceMemory: resource.MustParse(isvc.Spec.Resources.Memory),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\treturn deployment\n}\n\nfunc (r *InferenceServiceReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&inferencev1alpha1.InferenceService{}).\n\t\tOwns(&appsv1.Deployment{}). // Watch Deployments we create\n\t\tWithEventFilter(predicate.GenerationChangedPredicate{}). // Ignore status-only updates\n\t\tComplete(r)\n}\n```\n\n: Called when an InferenceService changes or a watched resource changes`Reconcile()`\n\n: Links Deployment to InferenceService (for cleanup)`SetControllerReference()`\n\n: Watch Deployments we create; if one changes, re-reconcile the InferenceService`Owns()`\n\n**Status subresource**: Updated separately with`r.Status().Update()`\n\n**RBAC**: The`+kubebuilder:rbac:`\n\ncomments generate Kubernetes RBAC rules\n\n-\n`inferenceservice_controller.go`\n\nimplements`Reconcile()`\n\n-\n`constructDeployment()`\n\ncreates valid k8s Deployment specs - Controller compiles:\n`make build`\n\n- RBAC rules generate:\n`make manifests`\n\n- Understand the reconciliation loop flow\n\n**Shell commands from Go**— How to run`kubectl`\n\nand`kind`\n\ncommands**Logging**— Using`log.FromContext(ctx)`\n\nand`log.Info()`\n\n**Testing with real Kubernetes**— Integration testing concepts** Environment setup**— How`KUBECONFIG`\n\nand kubectl contexts work\n\n- Set up a local Kubernetes cluster with\n`kind`\n\n- Deploy your operator locally\n- Manually test reconciliation\n- Debug with logs and kubectl\n\nCreate a local test environment where you can:\n\n- Deploy your operator\n- Create an InferenceService CR\n- Watch the controller reconcile it into a Deployment\n- Verify everything works\n\n```\n# Create a kind cluster\nkind create cluster --name inferno-dev\n\n# Verify it works\nkubectl get nodes\nkubectl get pods -A\n\n# Install your operator CRDs\nmake install\n\n# Deploy the operator\nmake deploy\n\n# Check it's running\nkubectl get deployment -n inferno-system\nkubectl logs -n inferno-system deployment/inferno-controller-manager -f\n```\n\nCreate `config/samples/inference_v1alpha1_inferenceservice.yaml`\n\n:\n\n```\napiVersion: inference.inferno.io/v1alpha1\nkind: InferenceService\nmetadata:\n  name: test-llama\nspec:\n  model:\n    uri: s3://models/llama-3\n    framework: fake\n    version: \"1\"\n  replicas:\n    min: 1\n    max: 3\n  resources:\n    cpu: \"2\"\n    memory: \"8Gi\"\n    gpu: \"0\"\n  autoscaling:\n    enabled: false\n  traffic:\n    - version: \"1\"\n      weight: 100\n```\n\nDeploy and observe:\n\n```\n# Apply the CR\nkubectl apply -f config/samples/inference_v1alpha1_inferenceservice.yaml\n\n# Watch the controller work\nkubectl logs -n inferno-system deployment/inferno-controller-manager -f\n\n# Check if Deployment was created\nkubectl get deployments\nkubectl get inferenceservices\nkubectl describe inferenceservice test-llama\n\n# Check the status\nkubectl get inferenceservice test-llama -o yaml\n# View controller logs\nkubectl logs -n inferno-system deployment/inferno-controller-manager --tail=100 -f\n\n# Describe what happened\nkubectl describe inferenceservice test-llama\nkubectl describe deployment test-llama\n\n# Check events\nkubectl get events --all-namespaces | grep inferno\n\n# Interactive debugging (if needed)\nkubectl exec -it deployment/inferno-controller-manager -n inferno-system -- bash\n```\n\n- kind cluster running locally\n- Operator deployed:\n`make deploy`\n\n- InferenceService CR created\n- Deployment automatically created from the CR\n- Status updated correctly\n- Logs show clean reconciliation\n\n**Helper functions**— Extracting common code into`constructService()`\n\n**Go interfaces as contracts**— Services follow same pattern as Deployments** Type conversion**— Using`intstr.FromString()`\n\nfor port types**Map creation**— Building label maps for selectors\n\n- Create and manage Kubernetes Services\n- Expose inference endpoints\n- Handle traffic routing\n- Understand label selectors\n\nWhen an InferenceService is created, the operator should also create a Kubernetes Service so clients can reach the inference server.\n\n```\nInferenceService (user writes this)\n       ↓\n   Controller\n       ├→ Deployment (pods run the server)\n       └→ Service (exposes the server)\n```\n\nUpdate `constructDeployment()`\n\nand add a new `constructService()`\n\nin your controller:\n\n```\n// In Reconcile(), after creating/updating Deployment:\n\n// Step 2b: Create or update Service\nservice := &corev1.Service{}\nserviceName := types.NamespacedName{\n\tName:      inferenceService.Name,\n\tNamespace: inferenceService.Namespace,\n}\n\nif err := r.Get(ctx, serviceName, service); err != nil {\n\tif apierrors.IsNotFound(err) {\n\t\tservice = r.constructService(&inferenceService)\n\t\tif err := controllerutil.SetControllerReference(&inferenceService, service, r.Scheme); err != nil {\n\t\t\tlog.Error(err, \"Failed to set service controller reference\")\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t\tif err := r.Create(ctx, service); err != nil {\n\t\t\tlog.Error(err, \"Failed to create Service\")\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t\tlog.Info(\"Created Service\", \"service\", service.Name)\n\t} else {\n\t\tlog.Error(err, \"Failed to fetch Service\")\n\t\treturn ctrl.Result{}, err\n\t}\n}\n\n// constructService builds a Service for the InferenceService\nfunc (r *InferenceServiceReconciler) constructService(isvc *inferencev1alpha1.InferenceService) *corev1.Service {\n\tlabels := map[string]string{\n\t\t\"app\":      isvc.Name,\n\t\t\"operator\": \"inferno\",\n\t}\n\n\tservice := &corev1.Service{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName:      isvc.Name,\n\t\t\tNamespace: isvc.Namespace,\n\t\t},\n\t\tSpec: corev1.ServiceSpec{\n\t\t\tSelector: labels,\n\t\t\tType:     corev1.ServiceTypeClusterIP,\n\t\t\tPorts: []corev1.ServicePort{\n\t\t\t\t{\n\t\t\t\t\tName:       \"http\",\n\t\t\t\t\tPort:       80,\n\t\t\t\t\tTargetPort: intstr.FromString(\"http\"),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\treturn service\n}\n# Deploy updated operator\nmake deploy\n\n# Create an InferenceService\nkubectl apply -f config/samples/inference_v1alpha1_inferenceservice.yaml\n\n# Verify Service was created\nkubectl get svc test-llama\nkubectl describe svc test-llama\n\n# Test from inside the cluster\nkubectl run -it --rm debug --image=curlimages/curl --restart=Never -- \\\n  curl http://test-llama/health\n```\n\n- Service created automatically with InferenceService\n- Service selects the right Pods\n- Labels and selectors match Deployment\n- Service exposed at DNS name\n`<name>.<namespace>.svc.cluster.local`\n\n- RBAC updated for Services:\n`make manifests`\n\n**Conditional creation**— When to create vs skip resources** Pointer arithmetic**— Working with`*int32`\n\nvalues**Default values**— Setting sensible defaults in helpers** Complex struct composition**— HPA specs have deeply nested structs\n\n- Create HorizontalPodAutoscaler (HPA) from the InferenceService\n- Handle min/max replicas\n- Understand metric-based scaling\n- Integrate with KEDA (future enhancement)\n\nWhen `autoscaling.enabled: true`\n\n, the operator should create an HPA that scales the Deployment:\n\n```\nInferenceService spec:\n  autoscaling:\n    enabled: true\n    targetConcurrency: 10\n    minReplicas: 1\n    maxReplicas: 5\n       ↓\n   HPA created\n       ↓\n   Metrics Server watches CPU/memory\n       ↓\n   Scales Deployment 1→5 based on load\n```\n\nIn your controller, add:\n\n```\n// After Service creation, in Reconcile():\n\nif inferenceService.Spec.Autoscaling.Enabled {\n\thpa := &autoscalingv2.HorizontalPodAutoscaler{}\n\thpaName := types.NamespacedName{\n\t\tName:      inferenceService.Name,\n\t\tNamespace: inferenceService.Namespace,\n\t}\n\n\tif err := r.Get(ctx, hpaName, hpa); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\thpa = r.constructHPA(&inferenceService)\n\t\t\tif err := controllerutil.SetControllerReference(&inferenceService, hpa, r.Scheme); err != nil {\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\t\t\tif err := r.Create(ctx, hpa); err != nil {\n\t\t\t\tlog.Error(err, \"Failed to create HPA\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\t\t\tlog.Info(\"Created HPA\", \"hpa\", hpa.Name)\n\t\t} else {\n\t\t\tlog.Error(err, \"Failed to fetch HPA\")\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t}\n}\n\n// constructHPA builds an HPA for the InferenceService\nfunc (r *InferenceServiceReconciler) constructHPA(isvc *inferencev1alpha1.InferenceService) *autoscalingv2.HorizontalPodAutoscaler {\n\tminReplicas := int32(isvc.Spec.Replicas.Min)\n\tmaxReplicas := int32(isvc.Spec.Replicas.Max)\n\n\t// Default to CPU-based scaling if no custom metric\n\tcpuUtilization := int32(80)\n\n\thpa := &autoscalingv2.HorizontalPodAutoscaler{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName:      isvc.Name,\n\t\t\tNamespace: isvc.Namespace,\n\t\t},\n\t\tSpec: autoscalingv2.HorizontalPodAutoscalerSpec{\n\t\t\tScaleTargetRef: autoscalingv2.CrossVersionObjectReference{\n\t\t\t\tAPIVersion: \"apps/v1\",\n\t\t\t\tKind:       \"Deployment\",\n\t\t\t\tName:       isvc.Name,\n\t\t\t},\n\t\t\tMinReplicas: &minReplicas,\n\t\t\tMaxReplicas: maxReplicas,\n\t\t\tMetrics: []autoscalingv2.MetricSpec{\n\t\t\t\t{\n\t\t\t\t\tType: autoscalingv2.ResourceMetricSourceType,\n\t\t\t\t\tResource: &autoscalingv2.ResourceMetricSource{\n\t\t\t\t\t\tName:                     corev1.ResourceCPU,\n\t\t\t\t\t\tTargetAverageUtilization: &cpuUtilization,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\treturn hpa\n}\n```\n\nAlso update RBAC:\n\n```\n// +kubebuilder:rbac:groups=autoscaling,resources=horizontalpodautoscalers,verbs=get;list;watch;create;update;patch;delete\n# Deploy\nmake deploy\n\n# Create an InferenceService with autoscaling\ncat <<EOF | kubectl apply -f -\napiVersion: inference.inferno.io/v1alpha1\nkind: InferenceService\nmetadata:\n  name: autoscale-test\nspec:\n  model:\n    uri: s3://models/test\n    framework: fake\n    version: \"1\"\n  replicas:\n    min: 1\n    max: 5\n  resources:\n    cpu: \"1\"\n    memory: \"1Gi\"\n  autoscaling:\n    enabled: true\n    targetConcurrency: 10\n    minReplicas: 1\n    maxReplicas: 5\nEOF\n\n# Check HPA was created\nkubectl get hpa\nkubectl describe hpa autoscale-test\n\n# (HPA needs metrics-server; kind doesn't have it by default, so skip load testing for now)\n```\n\n- HPA created when\n`autoscaling.enabled: true`\n\n- Min/max replicas respected\n- CPU-based scaling configured\n- HPA updates when InferenceService spec changes\n- RBAC includes HPA permissions\n\n**Receiver methods**— The`(r *Reconciler)`\n\npattern (object-oriented Go)**Slice manipulation**— Appending, finding in arrays of conditions** Boolean logic**— Conditional field setting** Time handling**—`time.Now()`\n\nand`metav1.Time`\n\ntypes**Defer for cleanup**— Resource management patterns\n\n- Implement proper status tracking\n- Use Kubernetes Conditions pattern\n- Handle error states gracefully\n- Implement retry logic with exponential backoff\n\nTrack the lifecycle of an InferenceService:\n\n```\nCreating → Pending → Ready ✓\n     ↓ (error)\n     Failed → Retry\n```\n\nUpdate your `InferenceServiceStatus`\n\nto track conditions:\n\n```\n// In controllers/inferenceservice_controller.go\n\nfunc (r *InferenceServiceReconciler) updateStatus(ctx context.Context, isvc *inferencev1alpha1.InferenceService, phase string, message string) error {\n\tlog := log.FromContext(ctx)\n\n\tisvc.Status.Phase = phase\n\tisvc.Status.Message = message\n\tisvc.Status.LastUpdateTime = &metav1.Time{Time: time.Now()}\n\n\t// Add condition\n\tcondition := metav1.Condition{\n\t\tType:               \"Ready\",\n\t\tStatus:             metav1.ConditionTrue,\n\t\tObservedGeneration: isvc.Generation,\n\t\tReason:             phase,\n\t\tMessage:            message,\n\t\tLastTransitionTime: metav1.Time{Time: time.Now()},\n\t}\n\n\tif phase != \"Running\" {\n\t\tcondition.Status = metav1.ConditionFalse\n\t}\n\n\t// Upsert condition (replace if exists, append if not)\n\tfound := false\n\tfor i, c := range isvc.Status.Conditions {\n\t\tif c.Type == condition.Type {\n\t\t\tisvc.Status.Conditions[i] = condition\n\t\t\tfound = true\n\t\t\tbreak\n\t\t}\n\t}\n\tif !found {\n\t\tisvc.Status.Conditions = append(isvc.Status.Conditions, condition)\n\t}\n\n\tif err := r.Status().Update(ctx, isvc); err != nil {\n\t\tlog.Error(err, \"Failed to update status\")\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\n// In Reconcile(), use it like:\nif err := r.updateStatus(ctx, &inferenceService, \"Running\", \"Deployment is ready\"); err != nil {\n\treturn ctrl.Result{}, err\n}\n\n// For errors, retry with backoff:\nif err != nil {\n\tr.updateStatus(ctx, &inferenceService, \"Error\", err.Error())\n\treturn ctrl.Result{RequeueAfter: 10 * time.Second}, nil // Retry in 10s\n}\nkubectl apply -f config/samples/inference_v1alpha1_inferenceservice.yaml\n\n# Watch status in real-time\nkubectl get inferenceservice -o wide -w\n\n# View conditions\nkubectl get inferenceservice test-llama -o yaml | grep -A 10 conditions\n\n# Create a bad spec to trigger error\nkubectl apply -f - <<EOF\napiVersion: inference.inferno.io/v1alpha1\nkind: InferenceService\nmetadata:\n  name: bad-service\nspec:\n  model:\n    uri: s3://models/test\n    framework: invalid  # This should fail validation\n    version: \"1\"\n  replicas:\n    min: 1\n    max: 5\nEOF\n```\n\n- Status updated on each reconciliation\n- Conditions follow Kubernetes pattern\n- Error states tracked\n- Retry logic with backoff implemented\n- kubectl can show readable status\n\n**Testing patterns**— Table-driven tests, subtests** Goroutine testing**— Using`Eventually()`\n\nfor async operations**Mocking with interfaces**— Using`client.Client`\n\ninterface for testing**BDD style testing**— Ginkgo framework (`Describe`\n\n,`It`\n\n,`Eventually`\n\n)**Test fixtures**— Setting up test data\n\n- Write unit tests for your controller\n- Write integration tests with envtest\n- Test edge cases and error handling\n- Achieve good test coverage\n\n```\n✓ Creating InferenceService → Deployment created\n✓ Updating spec → Deployment updated\n✓ Deleting InferenceService → Deployment deleted (via owner ref)\n✓ Invalid spec → Error state\n✓ Missing resources → Retry\n✓ Concurrent reconciliations → No race conditions\n```\n\nCreate `controllers/inferenceservice_controller_test.go`\n\n:\n\n```\npackage controllers\n\nimport (\n\t\"context\"\n\n\t. \"github.com/onsi/ginkgo/v2\"\n\t. \"github.com/onsi/gomega\"\n\n\tappsv1 \"k8s.io/api/apps/v1\"\n\tcorev1 \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\n\tinferencev1alpha1 \"github.com/yourusername/inferno/api/v1alpha1\"\n)\n\nvar _ = Describe(\"InferenceServiceReconciler\", func() {\n\tContext(\"Creating an InferenceService\", func() {\n\t\tIt(\"should create a Deployment\", func() {\n\t\t\tctx := context.Background()\n\n\t\t\tisvc := &inferencev1alpha1.InferenceService{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName:      \"test-service\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: inferencev1alpha1.InferenceServiceSpec{\n\t\t\t\t\tModel: inferencev1alpha1.ModelSpec{\n\t\t\t\t\t\tURI:       \"s3://models/test\",\n\t\t\t\t\t\tFramework: \"fake\",\n\t\t\t\t\t\tVersion:   \"1\",\n\t\t\t\t\t},\n\t\t\t\t\tReplicas: inferencev1alpha1.ReplicaConfig{\n\t\t\t\t\t\tMin: 1,\n\t\t\t\t\t\tMax: 3,\n\t\t\t\t\t},\n\t\t\t\t\tResources: inferencev1alpha1.ResourceRequirements{\n\t\t\t\t\t\tCPU:    \"1\",\n\t\t\t\t\t\tMemory: \"1Gi\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Create the InferenceService\n\t\t\tExpect(k8sClient.Create(ctx, isvc)).Should(Succeed())\n\n\t\t\t// Eventually, a Deployment should exist\n\t\t\tdeployment := &appsv1.Deployment{}\n\t\t\tdeploymentName := types.NamespacedName{Name: \"test-service\", Namespace: \"default\"}\n\t\t\tEventually(func() error {\n\t\t\t\treturn k8sClient.Get(ctx, deploymentName, deployment)\n\t\t\t}).Should(Succeed())\n\n\t\t\t// Verify the Deployment spec\n\t\t\tExpect(*deployment.Spec.Replicas).To(Equal(int32(1)))\n\t\t\tExpect(deployment.Spec.Template.Spec.Containers[0].Name).To(Equal(\"inference-server\"))\n\t\t})\n\n\t\tIt(\"should create a Service\", func() {\n\t\t\tctx := context.Background()\n\n\t\t\tisvc := &inferencev1alpha1.InferenceService{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName:      \"test-svc-2\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: inferencev1alpha1.InferenceServiceSpec{\n\t\t\t\t\tModel: inferencev1alpha1.ModelSpec{\n\t\t\t\t\t\tURI:       \"s3://models/test\",\n\t\t\t\t\t\tFramework: \"fake\",\n\t\t\t\t\t\tVersion:   \"1\",\n\t\t\t\t\t},\n\t\t\t\t\tReplicas: inferencev1alpha1.ReplicaConfig{Min: 1, Max: 3},\n\t\t\t\t\tResources: inferencev1alpha1.ResourceRequirements{\n\t\t\t\t\t\tCPU:    \"1\",\n\t\t\t\t\t\tMemory: \"1Gi\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tExpect(k8sClient.Create(ctx, isvc)).Should(Succeed())\n\n\t\t\t// Eventually, a Service should exist\n\t\t\tservice := &corev1.Service{}\n\t\t\tserviceName := types.NamespacedName{Name: \"test-svc-2\", Namespace: \"default\"}\n\t\t\tEventually(func() error {\n\t\t\t\treturn k8sClient.Get(ctx, serviceName, service)\n\t\t\t}).Should(Succeed())\n\n\t\t\tExpect(service.Spec.Selector[\"app\"]).To(Equal(\"test-svc-2\"))\n\t\t})\n\t})\n\n\tContext(\"Updating an InferenceService\", func() {\n\t\tIt(\"should update the Deployment replicas\", func() {\n\t\t\tctx := context.Background()\n\n\t\t\tisvc := &inferencev1alpha1.InferenceService{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName:      \"update-test\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: inferencev1alpha1.InferenceServiceSpec{\n\t\t\t\t\tModel:     inferencev1alpha1.ModelSpec{URI: \"s3://models/test\", Framework: \"fake\", Version: \"1\"},\n\t\t\t\t\tReplicas:  inferencev1alpha1.ReplicaConfig{Min: 1, Max: 3},\n\t\t\t\t\tResources: inferencev1alpha1.ResourceRequirements{CPU: \"1\", Memory: \"1Gi\"},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tExpect(k8sClient.Create(ctx, isvc)).Should(Succeed())\n\n\t\t\t// Update replica count\n\t\t\tisvc.Spec.Replicas.Min = 2\n\t\t\tExpect(k8sClient.Update(ctx, isvc)).Should(Succeed())\n\n\t\t\t// Verify Deployment was updated\n\t\t\tdeployment := &appsv1.Deployment{}\n\t\t\tdeploymentName := types.NamespacedName{Name: \"update-test\", Namespace: \"default\"}\n\t\t\tEventually(func(int32) {\n\t\t\t\tk8sClient.Get(ctx, deploymentName, deployment)\n\t\t\t\treturn *deployment.Spec.Replicas\n\t\t\t}).Should(Equal(int32(2)))\n\t\t})\n\t})\n\n\tContext(\"Deleting an InferenceService\", func() {\n\t\tIt(\"should delete owned Deployment and Service\", func() {\n\t\t\tctx := context.Background()\n\n\t\t\tisvc := &inferencev1alpha1.InferenceService{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName:      \"delete-test\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: inferencev1alpha1.InferenceServiceSpec{\n\t\t\t\t\tModel:     inferencev1alpha1.ModelSpec{URI: \"s3://models/test\", Framework: \"fake\", Version: \"1\"},\n\t\t\t\t\tReplicas:  inferencev1alpha1.ReplicaConfig{Min: 1, Max: 3},\n\t\t\t\t\tResources: inferencev1alpha1.ResourceRequirements{CPU: \"1\", Memory: \"1Gi\"},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tExpect(k8sClient.Create(ctx, isvc)).Should(Succeed())\n\n\t\t\t// Delete the InferenceService\n\t\t\tExpect(k8sClient.Delete(ctx, isvc)).Should(Succeed())\n\n\t\t\t// Verify Deployment is deleted (cascade delete via owner ref)\n\t\t\tdeployment := &appsv1.Deployment{}\n\t\t\tdeploymentName := types.NamespacedName{Name: \"delete-test\", Namespace: \"default\"}\n\t\t\tEventually(func() bool {\n\t\t\t\terr := k8sClient.Get(ctx, deploymentName, deployment)\n\t\t\t\treturn err != nil // Should not exist\n\t\t\t}).Should(BeTrue())\n\t\t})\n\t})\n})\n# Run all tests\nmake test\n\n# Run with coverage\ngo test ./... -coverprofile=coverage.out\ngo tool cover -html=coverage.out\n```\n\n- Unit tests for controller logic\n- Integration tests with envtest\n- Tests for create, update, delete\n- Error case handling tested\n- >80% code coverage\n\n**Doc comments**— Go's documentation convention (comments above functions/types)** Godoc**— Auto-generating documentation from code comments** YAML marshalling**— How Go structs become YAML examples** Code comments vs documentation**— When to explain the why, not what\n\n- Document the API (CRD)\n- Create user-facing examples\n- Document installation\n- Prepare for deployment\n\n**README.md**— Project overview** docs/API.md**— API reference** docs/DEPLOYMENT.md**— How to deploy** docs/EXAMPLES.md**— Real-world examples** Helm chart**(optional, for Day 10)\n\n**docs/API.md:**\n\n```\n# InferenceService API Reference\n\n## Spec\n\n### model (required)\n- `uri` (string): S3 path to the model\n- `framework` (string): `fake`, `vllm`, or `triton`\n- `version` (string): Model version tag\n\n### replicas (required)\n- `min` (int): Minimum pods\n- `max` (int): Maximum pods\n\n### resources (optional)\n- `cpu` (string): CPU request (e.g., \"2\")\n- `memory` (string): Memory request (e.g., \"8Gi\")\n- `gpu` (string): Number of GPUs (e.g., \"1\")\n\n### autoscaling (optional)\n- `enabled` (bool): Enable HPA\n- `targetConcurrency` (int): Target requests per pod\n- `minReplicas` (int): HPA min (overrides replicas.min)\n- `maxReplicas` (int): HPA max (overrides replicas.max)\n\n### traffic (optional)\n- Array of `{version, weight}` for canary deployments\n\n## Status\n\n- `phase`: Pending | Running | Failed\n- `readyReplicas`: How many pods are ready\n- `conditions`: Array of Kubernetes conditions\n```\n\n**docs/DEPLOYMENT.md:**\n\n```\n# Deploying Inferno\n\n## Prerequisites\n- Kubernetes 1.19+\n- kind/minikube/real cluster\n\n## Installation\n\n``` bash\n# Clone and install\ngit clone github.com/yourusername/inferno\ncd inferno\n\n# Install CRDs\nmake install\n\n# Deploy operator\nmake deploy\n\n# Verify\nkubectl get deployment -n inferno-system\nkubectl apply -f config/samples/inference_v1alpha1_inferenceservice.yaml\nkubectl get inferenceservices\n### Deliverables\n- [ ] README.md with project description\n- [ ] docs/API.md with complete API reference\n- [ ] docs/DEPLOYMENT.md with setup instructions\n- [ ] docs/EXAMPLES.md with 3-5 real examples\n- [ ] All code has docstrings\n- [ ] CRD manifests are clean\n\n---\n\n## Day 10: Integration Testing & Final Polish\n\n### Go Concepts You'll Learn Today\n- **Bash scripting from Go** — Shelling out to run commands\n- **Error recovery** — Handling failures gracefully\n- **Cleanup patterns** — Teardown after tests\n- **Concurrency testing** — Race conditions and data races\n- **Real-world debugging** — Reading logs, tracing issues\n\n### Learning Objectives\n- End-to-end integration testing\n- Fix any remaining bugs\n- Prepare for production\n- Document next steps\n\n### What You'll Do\n\n1. **E2E Test**: Deploy operator → create InferenceService → verify all resources\n2. **Stress Test**: Create many InferenceServices, verify controller handles it\n3. **Failure Test**: Kill pods, verify operator recovers\n4. **Documentation**: Finalize docs, create troubleshooting guide\n\n### Hands-On: E2E Test Script\n\nCreate `e2e_test.sh`:\n\n``` bash\n#!/bin/bash\nset -e\n\necho \"=== Inferno E2E Test ===\"\n\n# Setup\nkind create cluster --name inferno-e2e || true\nkubectl config use-context kind-inferno-e2e\n\n# Deploy\nmake deploy\n\necho \"Waiting for operator to be ready...\"\nkubectl wait --for=condition=available --timeout=300s \\\n  deployment/inferno-controller-manager -n inferno-system\n\n# Create InferenceService\necho \"Creating InferenceService...\"\nkubectl apply -f config/samples/inference_v1alpha1_inferenceservice.yaml\n\n# Wait for Deployment\necho \"Waiting for Deployment...\"\nkubectl rollout status deployment/test-llama --timeout=120s\n\n# Verify Service\necho \"Verifying Service...\"\nkubectl get svc test-llama\nkubectl get endpoints test-llama\n\n# Check status\necho \"Checking status...\"\nkubectl get inferenceservice test-llama -o yaml\n\n# Test port-forward\necho \"Testing connectivity...\"\nkubectl port-forward svc/test-llama 8000:80 &\nsleep 2\ncurl localhost:8000/health || true\nkill %1\n\necho \"=== All tests passed ===\"\n\n# Cleanup\nkind delete cluster --name inferno-e2e\n# Create many InferenceServices\nfor i in {1..10}; do\n  kubectl apply -f - <<EOF\napiVersion: inference.inferno.io/v1alpha1\nkind: InferenceService\nmetadata:\n  name: stress-test-$i\nspec:\n  model:\n    uri: s3://models/test\n    framework: fake\n    version: \"1\"\n  replicas:\n    min: 1\n    max: 3\n  resources:\n    cpu: \"1\"\n    memory: \"1Gi\"\nEOF\ndone\n\n# Verify all were created\nkubectl get inferenceservices\nkubectl get deployments\n\n# Check controller logs for errors\nkubectl logs -n inferno-system deployment/inferno-controller-manager\n# Create an InferenceService\nkubectl apply -f config/samples/inference_v1alpha1_inferenceservice.yaml\n\n# Kill the pod\nkubectl delete pod -l app=test-llama\n\n# Verify operator recreates it\nkubectl get pods -l app=test-llama\nkubectl wait --for=condition=ready pod -l app=test-llama\n\necho \"Pod recovered successfully\"\n```\n\n- E2E test script passes\n- Stress test with 10+ InferenceServices works\n- Failure recovery tested and documented\n- All logs clean (no errors)\n- README complete with quick-start\n- API docs comprehensive\n- Code well-commented\n- RBAC rules correct\n- CRD validation rules correct\n\n**Replace fake server** with vLLM container image**Add Prometheus metrics** for monitoring**Implement canary deployments** with traffic splitting**Add webhooks** for advanced validation**Create Helm chart** for easy deployment**GPU support** with node affinity**Multi-model serving** with routing**Model caching** layer**Integration with KEDA** for custom metrics**Security hardening** and network policies\n\nCopy this into a todo tracker:\n\n```\nDay 1: Kubernetes Controllers & Kubebuilder\n  ☐ Understand reconciliation loop\n  ☐ Initialize kubebuilder project\n  ☐ Create InferenceService CRD scaffold\n  ☐ Project compiles\n\nDay 2: API Schema\n  ☐ Define ModelSpec, ReplicaConfig, AutoscalingSpec\n  ☐ Add validation rules\n  ☐ Implement status subresource\n  ☐ Generate CRD manifests\n  ☐ Validate YAML deploys\n\nDay 3: Reconciliation Logic\n  ☐ Implement Reconcile() function\n  ☐ Create constructDeployment()\n  ☐ Add controller reference for cleanup\n  ☐ Update status correctly\n  ☐ Controller compiles and deploys\n\nDay 4: Local Development\n  ☐ Create kind cluster\n  ☐ Deploy operator locally\n  ☐ Create test InferenceService\n  ☐ Verify Deployment created\n  ☐ Debug with logs\n\nDay 5: Service Creation\n  ☐ Implement constructService()\n  ☐ Create Service alongside Deployment\n  ☐ Update RBAC for Services\n  ☐ Test service creation\n  ☐ Verify DNS name works\n\nDay 6: Autoscaling\n  ☐ Implement HPA creation logic\n  ☐ Support min/max replicas\n  ☐ Add autoscaling validation\n  ☐ Test HPA creation\n  ☐ Verify scaling behavior\n\nDay 7: Status & Conditions\n  ☐ Implement updateStatus() helper\n  ☐ Add Kubernetes conditions\n  ☐ Implement retry logic\n  ☐ Handle error states\n  ☐ Test status updates\n\nDay 8: Testing\n  ☐ Write unit tests\n  ☐ Write integration tests with envtest\n  ☐ Test create/update/delete\n  ☐ Test error cases\n  ☐ Achieve >80% coverage\n\nDay 9: Documentation\n  ☐ Write README.md\n  ☐ Document API in docs/API.md\n  ☐ Write deployment guide\n  ☐ Create example YAML files\n  ☐ Add troubleshooting guide\n\nDay 10: Integration & Polish\n  ☐ Create E2E test script\n  ☐ Stress test with 10+ services\n  ☐ Test failure recovery\n  ☐ Fix any remaining bugs\n  ☐ Final code review and cleanup\n```\n\nBy Day 10, you should have:\n\n✅ **Working Kubernetes Operator** that:\n\n- Watches InferenceService CRs\n- Creates Deployments, Services, HPAs automatically\n- Updates status correctly\n- Handles errors and retries\n- Passes integration tests\n\n✅ **Code Quality**:\n\n-\n80% test coverage\n\n- Clean logs (no errors/warnings)\n- Comprehensive error handling\n- Well-documented API\n\n✅ **Documentation**:\n\n- User-friendly README\n- Complete API reference\n- Deployment guide\n- Real-world examples\n\n✅ **Ready for Next Phase**:\n\n- Can replace fake server with real vLLM\n- Can add monitoring/observability\n- Can scale to production\n\n**Go & Kubernetes:**\n\n**Operators & CRDs:**\n\n**Tools:**\n\n**Testing:**\n\nYou won't use all of Go, but you'll use these patterns *constantly*:\n\n```\n// EVERY function call needs this:\nif err != nil {\n    log.Error(err, \"what went wrong\")\n    return ctrl.Result{}, err\n}\n\n// For optional errors:\nif err != nil {\n    if apierrors.IsNotFound(err) {\n        // Handle not found\n    } else {\n        return ctrl.Result{}, err\n    }\n}\n\n// Wrapping errors with context:\nif err != nil {\n    return fmt.Errorf(\"failed to create deployment: %w\", err)\n}\n// Creating a pointer to an int\ncount := int32(5)\nptr := &count          // Get address\nvalue := *ptr          // Dereference\n\n// In structs, K8s uses pointers for optional fields:\nspec.Replicas = &count // Optional field\n*spec.Replicas         // Read it\n\n// Nil-safe access:\nif spec.Replicas != nil {\n    count = *spec.Replicas\n}\n// Attach a method to a struct\ntype MyReconciler struct {\n    client.Client\n    Scheme *runtime.Scheme\n}\n\n// The (r *MyReconciler) is the \"receiver\" — like \"this\" in Java\nfunc (r *MyReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n    // Can access r.Client, r.Scheme\n    err := r.Get(ctx, req.NamespacedName, &obj)\n}\n// An interface says \"anything that has these methods\"\ntype Reader interface {\n    Read(p []byte) (n int, err error)\n}\n\n// k8s client.Client is an interface:\ntype Client interface {\n    Get(ctx, key, obj)\n    Create(ctx, obj)\n    Update(ctx, obj)\n    Delete(ctx, obj)\n}\n\n// Your code takes interfaces, not concrete types:\nfunc MyFunction(client client.Client) {\n    client.Get(...)  // Works with any client.Client\n}\n// Tags are metadata on struct fields\ntype MyStruct struct {\n    // JSON: how to encode as JSON/YAML\n    Name string `json:\"name\"`\n    \n    // Multiple tags:\n    Count int `json:\"count\" kubebuilder:\"validation:Minimum=1\"`\n    \n    // Omit empty values:\n    Optional string `json:\"optional,omitempty\"`\n    \n    // In k8s, kubebuilder tags control CRD generation:\n    // +kubebuilder:validation:Required\n    // +kubebuilder:validation:Enum=value1;value2\n    // +kubebuilder:rbac:groups=core,resources=pods,verbs=get;list\n}\njs\n// Create a slice\nvar items []string\n\n// Append to slice\nitems = append(items, \"new item\")\n\n// Iterate\nfor i, item := range items {\n    fmt.Println(i, item)\n}\n\n// Find in slice\nfor _, item := range items {\n    if item == \"target\" {\n        // Found it\n    }\n}\n\n// With maps (finding, replacing):\nconditions := []metav1.Condition{}\nfor i, c := range conditions {\n    if c.Type == \"Ready\" {\n        conditions[i] = newCondition  // Replace\n        found = true\n    }\n}\nif !found {\n    conditions = append(conditions, newCondition)  // Add\n}\n// Context flows through your code\nfunc (r *Reconciler) Reconcile(ctx context.Context, req Request) {\n    // ctx can be cancelled at any time\n    // Always check for cancellation:\n    \n    select {\n    case <-ctx.Done():\n        return ctx.Err()  // Was cancelled\n    default:\n        // Keep going\n    }\n    \n    // Timeouts:\n    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n    defer cancel()  // Clean up\n    \n    err := r.Get(ctx, name, obj)  // Will timeout after 10s\n}\n// Simple assertion\nif got != expected {\n    t.Errorf(\"got %v, want %v\", got, expected)\n}\n\n// Eventually (for async operations)\nEventually(func() bool {\n    obj := &MyObject{}\n    err := client.Get(ctx, name, obj)\n    return err == nil  // Keep checking until true\n}).Should(BeTrue())\n\n// Ginkgo-style (BDD)\nDescribe(\"MyController\", func() {\n    It(\"should create a Deployment\", func() {\n        // Test code\n        Expect(actual).To(Equal(expected))\n    })\n})\n// Create a map\nlabels := make(map[string]string)\nlabels[\"app\"] = \"myapp\"\nlabels[\"version\"] = \"1.0\"\n\n// Or with literal syntax:\nlabels := map[string]string{\n    \"app\":     \"myapp\",\n    \"version\": \"1.0\",\n}\n\n// Access\nv := labels[\"app\"]  // \"myapp\"\nv := labels[\"nonexistent\"]  // \"\" (zero value)\n\n// Check if key exists:\nif val, ok := labels[\"app\"]; ok {\n    // Key exists, val has the value\n}\n\n// Iterate\nfor key, value := range labels {\n    fmt.Println(key, value)\n}\nimport \"time\"\n\nnow := time.Now()\n\n// Create k8s time:\ntimestamp := metav1.Time{Time: time.Now()}\n\n// Calculate duration:\nfuture := now.Add(10 * time.Second)\nduration := future.Sub(now)  // 10 seconds\n\n// Requeue delays:\nreturn ctrl.Result{RequeueAfter: 5 * time.Second}, nil\n```\n\nCommon commands you'll use daily:\n\n```\n# Project setup (do this once)\nkubebuilder init --domain myorg.io --repo github.com/me/myop\nkubebuilder create api --group mygroup --version v1alpha1 --kind MyResource\n\n# Generate manifests (do this often)\nmake generate          # Generate code from struct tags\nmake manifests         # Generate CRD/RBAC/manager YAML\n\n# Local development\nmake install           # Install CRDs into current cluster\nmake uninstall         # Remove CRDs\nmake run              # Run controller locally (outside k8s)\nmake deploy           # Deploy controller to cluster\nmake undeploy         # Remove controller\n\n# Testing\nmake envtest          # Set up test environment\ngo test ./...         # Run all tests\ngo test -v ./...      # Verbose test output\ngo test -cover ./...  # Show coverage\n\n# Cleanup\nmake clean            # Remove generated files\n```\n\nEvery Kubernetes resource has this pattern:\n\n```\n// TypeMeta + ObjectMeta (required by all k8s resources)\ntype MyResource struct {\n    metav1.TypeMeta   `json:\",inline\"`          // apiVersion, kind\n    metav1.ObjectMeta `json:\"metadata,omitempty\"` // name, namespace, labels, etc\n    \n    Spec   MyResourceSpec   `json:\"spec,omitempty\"`     // What user wants\n    Status MyResourceStatus `json:\"status,omitempty\"`   // What actually exists\n}\n\n// List version (also required)\ntype MyResourceList struct {\n    metav1.TypeMeta `json:\",inline\"`\n    metav1.ListMeta `json:\"metadata,omitempty\"`\n    Items           []MyResource `json:\"items\"`\n}\n// Status is separate from Spec (important!)\ntype MyResourceStatus struct {\n    // Phase: the current state\n    Phase string `json:\"phase,omitempty\"`  // Pending, Running, Failed\n    \n    // Conditions: detailed state tracking\n    Conditions []metav1.Condition `json:\"conditions,omitempty\"`\n    \n    // ObservedGeneration: for detecting stale status\n    ObservedGeneration int64 `json:\"observedGeneration\"`\n}\n\n// Update it separately:\nr.Status().Update(ctx, resource)  // NOT r.Update()\nfunc (r *MyReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n    log := log.FromContext(ctx)\n    \n    // 1. Fetch the resource\n    myresource := &myv1.MyResource{}\n    if err := r.Get(ctx, req.NamespacedName, myresource); err != nil {\n        if apierrors.IsNotFound(err) {\n            return ctrl.Result{}, nil  // Deleted, nothing to do\n        }\n        return ctrl.Result{}, err  // Real error\n    }\n    \n    // 2. Check desired state exists\n    childResource := &SomeChildResource{}\n    err := r.Get(ctx, childName, childResource)\n    if err != nil {\n        if apierrors.IsNotFound(err) {\n            // 3a. Create it\n            childResource = r.constructChild(myresource)\n            controllerutil.SetControllerReference(myresource, childResource, r.Scheme)\n            if err := r.Create(ctx, childResource); err != nil {\n                log.Error(err, \"Failed to create child\")\n                return ctrl.Result{}, err\n            }\n        } else {\n            return ctrl.Result{}, err\n        }\n    } else {\n        // 3b. Update it\n        childResource = r.constructChild(myresource)\n        if err := r.Update(ctx, childResource); err != nil {\n            log.Error(err, \"Failed to update child\")\n            return ctrl.Result{}, err\n        }\n    }\n    \n    // 4. Update status\n    myresource.Status.ObservedGeneration = myresource.Generation\n    myresource.Status.Phase = \"Ready\"\n    if err := r.Status().Update(ctx, myresource); err != nil {\n        log.Error(err, \"Failed to update status\")\n        return ctrl.Result{}, err\n    }\n    \n    return ctrl.Result{}, nil\n}\n```\n\n**Good luck! Build Inferno day by day, test often, and enjoy the journey.** 🚀", "url": "https://wpnews.pro/news/inferno-kubernetes-native-ai-inference-operator-10-day-expert-learning-plan", "canonical_source": "https://gist.github.com/mwaykole/c78942324691d872f318806e65658fc5", "published_at": "2026-08-24 06:59:43+00:00", "updated_at": "2026-08-24 07:13:13.782789+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "mlops"], "entities": ["Kubebuilder", "Kubernetes", "Go"], "alternates": {"html": "https://wpnews.pro/news/inferno-kubernetes-native-ai-inference-operator-10-day-expert-learning-plan", "markdown": "https://wpnews.pro/news/inferno-kubernetes-native-ai-inference-operator-10-day-expert-learning-plan.md", "text": "https://wpnews.pro/news/inferno-kubernetes-native-ai-inference-operator-10-day-expert-learning-plan.txt", "jsonld": "https://wpnews.pro/news/inferno-kubernetes-native-ai-inference-operator-10-day-expert-learning-plan.jsonld"}}