The Remote Break-Even: Measuring When a Free Server Seat Beats a Local Agent

A refactor begins at 09:40 on a laptop that is already warm from a test run, and the first streamed token arrives about 1.2 seconds later. Six hundred tokens of patch follow at roughly 18 tokens per second, so the turn closes in about half a minute with nothing leaving the disk. Twenty minutes later the same loop is handed an 8 MB context bundle, and the same lane needs several minutes to answer. The interesting question is not whether local execution is virtuous, but whether a particular turn crosses a break-even point that can be measured before it runs.

Client-side inference keeps resurfacing as a topic, and current developer chatter about AI fatigue pushes the discussion toward ideology rather than arithmetic. The practical decision has three inputs: measured latency on the machine in front of you, the size of whatever must move across the network, and whether the payload is allowed to move at all. This article builds a small harness around those inputs, using a free remote seat as the second lane.

Note: the snippets below are illustrative and were not executed for this article. Every placeholder must be replaced with output from your own probe before any verdict is trusted.

The local lane is short: prefill time plus output tokens multiplied by per-token decode time. The remote lane is longer and much easier to underestimate, because it contains four terms that never appear in a vendor's headline throughput number. Those terms are round-trip and queue delay, upload time for the prompt payload, prompt prefill on the remote side, and download time for the response.

A fifth term matters more than the other four in agent workloads, and it is the handoff tax. Handoff includes serializing the current state, re-authenticating, rebuilding session context, and re-establishing whatever tool handles the loop had open. Teams routinely measure remote decode speed and then quietly pay a multi-second tax on every turn, which is why the arithmetic below treats handoff as a first-class cost rather than an implementation detail.

Run the probe on the machine that would actually host the local lane, because thermal state and memory pressure change these numbers by a wide margin. Take the median across at least five trials and keep the tail, since a single 4-second prefill spikes changes the verdict for short turns.

import statistics, time

def measure_local(call_local, prompt_tokens: int, out_tokens: int, trials: int = 5):
    """call_local(prompt_tokens, out_tokens) returns an iterator of token chunks."""
    prefill_ms, per_token_ms = [], []
    for _ in range(trials):
        t0 = time.perf_counter()
        stream = call_local(prompt_tokens, out_tokens)
        first = next(stream)                 # time to first token == prefill cost
        t1 = time.perf_counter()
        rest = list(stream)
        t2 = time.perf_counter()
        prefill_ms.append((t1 - t0) * 1000)
        per_token_ms.append(((t2 - t1) / max(1, len(rest))) * 1000)
    return {
        "prefill_s": statistics.median(prefill_ms) / 1000,
        "per_token_s": statistics.median(per_token_ms) / 1000,
        "prefill_p95_s": sorted(prefill_ms)[int(trials * 0.95) - 1] / 1000,
    }

Record the numbers as a local profile file and commit it next to the harness. A profile is only valid for one machine class and one prompt shape, and treating it as portable between laptops is how the break-even calculation quietly becomes fiction. Re-measure whenever the runtime, quantization, or available RAM changes.

With one local profile and one remote profile, the lane decision becomes subtraction. The calculator below takes placeholder values so the shape of the result is visible; substitute probe output and real network measurements before use.

from dataclasses import dataclass

@dataclass
class LocalProfile:
    prefill_s: float          # from measure_local()
    per_token_s: float        # seconds per generated token

@dataclass
class RemoteProfile:
    rtt_s: float              # round trip plus queueing
    prefill_tokens_per_s: float
    decode_tokens_per_s: float
    uplink_bytes_per_s: float
    downlink_bytes_per_s: float
    handoff_s: float          # serialize, re-auth, rebuild session

def estimate_local(p: LocalProfile, out_tokens: int) -> float:
    return p.prefill_s + out_tokens * p.per_token_s

def estimate_remote(r: RemoteProfile, prompt_tokens: int, out_tokens: int,
                    upload_bytes: int, download_bytes: int) -> float:
    return (r.rtt_s
            + upload_bytes / r.uplink_bytes_per_s
            + prompt_tokens / r.prefill_tokens_per_s
            + out_tokens / r.decode_tokens_per_s
            + download_bytes / r.downlink_bytes_per_s
            + r.handoff_s)

def verdict(local_s: float, remote_s: float, margin: float = 0.15) -> str:
    if remote_s < local_s * (1 - margin):
        return "remote"       # clear win, worth paying the handoff
    if local_s < remote_s * (1 - margin):
        return "local"
    return "either"           # inside the noise band; prefer local for secrets

Three scenarios show how sharply the answer moves when payload size changes. All numbers below are placeholders, not benchmarks: local at 1.2 s prefill and 18 tokens/second, remote at 55 decode tokens/second with 3.3 s of prefill for a 3,000-token prompt.

The middle row is the one that surprises people, because the upload of 8 MB at 1 Mbit/s costs about 67 seconds by itself. The third row shows why handoff deserves its own term: a 20-second state rebuild erases a 19-second remote advantage almost exactly. When a verdict lands in the noise band, the tie-break should come from data sensitivity rather than from a coin flip.

A latency verdict must never be able to move secret-bearing material across a network boundary. The check below runs first, and it is deliberately blunt: any field classified as secret pins the turn to the local lane regardless of what the arithmetic says.

RANK = {"public": 0, "internal": 1, "secret": 2}

def allowed_lane(fields, connectivity: bool, verdict_str: str) -> str:
    worst = max(RANK[f["class"]] for f in fields)
    if worst >= RANK["secret"]:
        return "local"          # no remote seat, whatever the latency math says
    if not connectivity:
        return "local"          # offline is a lane, not a failure
    return verdict_str

assert allowed_lane([{"class": "secret"}], True, "remote") == "local"
assert allowed_lane([{"class": "public"}], False, "remote") == "local"
assert allowed_lane([{"class": "internal"}], True, "remote") == "remote"

Classification is the hard part, and it is a review problem rather than a code problem. Marking a field as internal when it contains a customer identifier turns the gate into decoration, so the field list should be reviewed whenever a new tool is added to the loop. Redaction can downgrade a payload, but redaction has to happen before the gate reads the classes, not after the request is built.

A table keeps the policy reviewable by people who will never read the Python. The rows below are the ones that came up most often when auditing a mixed local and remote agent loop.

The rule of thumb that falls out of the table: remote wins on small payloads and long outputs, and local wins on large payloads and short outputs. Agent turns tend toward large payloads and short outputs, which is why a local-first default survives scrutiny more often than it fails it.

A remote lane is most useful before you own any remote lane, because that is when the probe has nothing to measure against. MonkeyCode, an open-source coding agent project, is described by its operator as offering free model access with a 10 million token allowance and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat both statements as operator-stated availability claims rather than benchmarks, and confirm the current terms on the project's own page before wiring anything automated to them. Under the placeholder arithmetic above, a turn consuming roughly 3,600 tokens works out to about 2,700 turns across a 10,000,000-token allowance, which is enough runway to run the probe and observe a real break-even rather than an assumed one. Free model access also makes the sensitivity gate cheap to test, since pinning a turn to local costs nothing but wall-clock time.

Two limits are worth stating plainly. The arithmetic in this article does not compare output quality, and a lane that returns a malformed schema is not faster at any latency. Quota and availability are variables in someone else's roadmap, so a pipeline that fails hard when the allowance changes has a dependency problem, not a capacity problem.

Latency figures vary by machine, network, and time of day, and the handoff tax varies by how much state your loop carries. The method survives that variance because it measures locally and recomputes; the specific numbers in this article do not transfer to your laptop.

A verdict that is never recorded cannot be reviewed when behavior changes. Append one JSON line per turn with the measured inputs, the chosen lane, and the gate result, then sample those lines weekly to see whether the break-even has drifted. The log is also the fastest way to discover that a previously internal field has quietly become secret-bearing.

{"turn": 412, "out_tokens": 600, "upload_bytes": 24000, "local_est_s": 34.5,
 "remote_est_s": 15.8, "worst_class": "internal", "lane": "remote"}

The instrument is the useful artifact here, not any single verdict. A laptop answers short turns offline and hands large bundles to a free seat when the upload cost is small, and the only way to know which case you are in is to measure it.