# Before Adding Gemma 4 to MonkeyCode, Run a Model Capability Contract

> Source: <https://dev.to/kongkong1/before-adding-gemma-4-to-monkeycode-run-a-model-capability-contract-4d07>
> Published: 2026-07-14 04:22:25+00:00

Gemma 4 is arriving in model catalogs with an unusually broad capability surface. Google's [official overview](https://ai.google.dev/gemma/docs/core), last updated July 8, 2026, lists five sizes, text and image input across the family, audio on selected variants, native system-role support, and 128K or 256K context windows depending on model size.

Those are model-family properties. They do **not** prove that a particular hosted endpoint, quantization, API adapter, or AI coding platform exposes every property correctly.

That distinction matters when adding a model to [MonkeyCode](https://github.com/chaitin/MonkeyCode), an AGPL-3.0 open-source AI development platform for teams. Its public repository describes managed server-side development environments, AI model and task management, project requirements, online use, native mobile clients, and private deployment.

This article reviews MonkeyCode at commit [ c58bcd4](https://github.com/chaitin/MonkeyCode/tree/c58bcd4dd4b7031f469a1271f276d22550b8f523) and builds the missing second gate: a reusable model capability contract.

It is a source review and test harness, **not** a Gemma 4 benchmark or a claim that MonkeyCode currently ships a verified Gemma 4 integration. I validated the harness against a local OpenAI-compatible fixture, not a live Gemma 4 endpoint.

MonkeyCode is relevant here because model choice is a first-class platform concern rather than a hard-coded SDK call.

At the reviewed commit, its [model configuration contract](https://github.com/chaitin/MonkeyCode/blob/c58bcd4dd4b7031f469a1271f276d22550b8f523/backend/domain/model.go#L198-L213) records:

The add-model flow then performs a health check before saving. That is the right first gate: reject a bad URL, credential, model ID, or protocol choice early.

The important limitation is explicit in [the health-check source](https://github.com/chaitin/MonkeyCode/blob/c58bcd4dd4b7031f469a1271f276d22550b8f523/backend/pkg/llm/check.go#L34-L36): it checks API reachability, authentication, and model existence, not answer content. For an OpenAI Chat endpoint, it sends `hi`

with `max_tokens: 1`

and accepts a non-error response.

That proves transport health. A coding agent needs more.

I would separate readiness into four gates:

| Gate | Question | Failure response |
|---|---|---|
| Transport | Can the platform authenticate and reach the exact model ID? | Do not save or route traffic |
| Protocol | Do system roles, streaming, tool calls, cancellation, and errors match the selected API contract? | Keep the model disabled |
| Capability | Does this deployed variant actually support the modalities and limits you advertise? | Remove the capability flag or change the route |
| Task quality | Does it pass your repository-specific coding evaluation within latency and cost budgets? | Do not make it a default |

MonkeyCode's current check covers the first gate. The next script exercises three protocol behaviors that a one-token response cannot establish.

Save this as `probe-openai-model.mjs`

. It requires Node.js 20 or newer and never prints the API key.

``` js
const required = ["MODEL_BASE_URL", "MODEL_API_KEY", "MODEL_ID"];
for (const name of required) {
  if (!process.env[name]) throw new Error(`Missing ${name}`);
}

const baseUrl = process.env.MODEL_BASE_URL.replace(/\/$/, "");
const endpoint = `${baseUrl}/chat/completions`;

async function chat(body) {
  const response = await fetch(endpoint, {
    method: "POST",
    headers: {
      authorization: `Bearer ${process.env.MODEL_API_KEY}`,
      "content-type": "application/json",
    },
    body: JSON.stringify({ model: process.env.MODEL_ID, ...body }),
    signal: AbortSignal.timeout(30_000),
  });
  const text = await response.text();
  if (!response.ok) throw new Error(`HTTP ${response.status}: ${text.slice(0, 300)}`);
  return { response, text };
}

const textResult = await chat({
  messages: [
    { role: "system", content: "Reply with exactly SYS_OK and nothing else." },
    { role: "user", content: "Follow the system instruction." },
  ],
  temperature: 0,
  max_tokens: 16,
});
const textBody = JSON.parse(textResult.text);
const content = textBody.choices?.[0]?.message?.content?.trim();
if (content !== "SYS_OK") {
  throw new Error(`system-role contract failed: ${JSON.stringify(content)}`);
}
console.log(`PASS system role — usage reported: ${Boolean(textBody.usage)}`);

const toolResult = await chat({
  messages: [{ role: "user", content: "Look up the weather for Paris." }],
  tools: [{
    type: "function",
    function: {
      name: "lookup_weather",
      description: "Look up weather by city",
      parameters: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"],
        additionalProperties: false,
      },
    },
  }],
  tool_choice: "required",
  max_tokens: 128,
});
const toolBody = JSON.parse(toolResult.text);
const call = toolBody.choices?.[0]?.message?.tool_calls?.[0];
if (call?.function?.name !== "lookup_weather") {
  throw new Error(`tool-call contract failed: ${toolResult.text.slice(0, 300)}`);
}
const args = JSON.parse(call.function.arguments);
if (args.city !== "Paris") throw new Error(`unexpected tool arguments: ${call.function.arguments}`);
console.log(`PASS tool call — ${call.function.name}`);

const streamResult = await chat({
  messages: [{ role: "user", content: "Reply with STREAM_OK." }],
  stream: true,
  temperature: 0,
  max_tokens: 16,
});
const contentType = streamResult.response.headers.get("content-type") || "";
if (!contentType.includes("text/event-stream")) {
  throw new Error(`stream contract failed: content-type was ${contentType || "missing"}`);
}
if (!streamResult.text.includes("data:") || !streamResult.text.includes("[DONE]")) {
  throw new Error(`stream contract failed: ${streamResult.text.slice(0, 300)}`);
}
console.log(`PASS SSE stream — ${streamResult.text.length} bytes`);
```

Run it against the **exact endpoint and model ID** you plan to configure:

```
MODEL_BASE_URL="https://provider.example/v1" \
MODEL_API_KEY="your-test-key" \
MODEL_ID="the-exact-provider-model-id" \
node probe-openai-model.mjs
```

The expected shape is:

```
PASS system role — usage reported: true
PASS tool call — lookup_weather
PASS SSE stream — 184 bytes
```

Treat that byte count as an example, not a benchmark. A provider may also omit usage from the response; the script reports that fact without failing the protocol gate.

The official Gemma 4 overview says small models have 128K context while medium models support 256K. It also warns that context length adds KV-cache memory beyond the static model weights.

So `context_limit: 256000`

should not be copied into MonkeyCode merely because the family documentation contains that number. Record a smaller verified operational envelope for the exact serving stack:

```
model_id: exact-provider-model-id
interface: openai_chat
system_role: pass
tool_calls: pass
streaming: pass
image_input: not_tested
audio_input: not_exposed_by_this_contract
declared_context_tokens: 256000
tested_context_tokens: 32000
tested_output_tokens: 4096
concurrent_requests: 4
timeout_seconds: 30
tested_at: 2026-07-14
```

This is deliberately conservative. A declared limit is documentation; a tested limit is evidence.

For a real coding rollout, add repository tasks after the protocol probe:

Run the same set against the current default model. Promote Gemma 4 only when a named variant, serving stack, and configuration meet predeclared thresholds. Do not infer coding quality from parameter count, context length, or one successful prompt.

MonkeyCode already provides useful control points for this workflow: central model configuration, explicit interface selection, capability flags, managed development environments, team task workflows, and private deployment. Because the project is open source, the exact health-check boundary can be inspected rather than guessed.

The practical improvement is to preserve that fast health check and add a versioned capability result beside it. Then a team can distinguish:

That is a safer way to adopt a fast-moving model family without slowing experimentation to a halt.

Disclosure: I contribute to the MonkeyCode project. The MonkeyCode observations above are based on the linked public repository at the specified commit. The probe was validated against a local fixture, not a live Gemma 4 deployment, and this article does not claim completed Gemma 4 compatibility or benchmark results.

If your team is evaluating model routing or private deployment, the [MonkeyCode Discord](https://discord.gg/2pPmuyr4pP) is the direct place to compare endpoint contracts and ask about currently supported configurations.