Remote Chat Failover Needs a Tab-Stop Origin Banner, Not a Skeleton Overlay

I was halfway through a keyboard-only pass on a streaming chat when the local path died. A skeleton overlay covered the composer, my focus jumped, and tokens started arriving from somewhere else. Did the interface tell me those tokens had left the browser? It did not, and that silence is the real defect.

Sighted teammates might notice a faint retry pulse and keep going. Keyboard and screen-reader users only felt a trap, then a voice that never named the new host. This write-up is a debugging retrospective, from that frozen caret to a typed origin state you can fail on purpose.

I built a small chat shell that retries when a local inference call rejects. The retry helper reused status: "streaming", so the UI never distinguished on-device work from a remote server. That collapse is how a privacy change becomes a animation, and it is an easy mistake when failover feels like infrastructure rather than UX.

Here is the interaction I treated as the failing transition.

If your table has one streaming row, you will hide origin changes from assistive tech. That is the pitfall, not the spinner’s CSS.

Why does a skeleton feel honest while still lying? Because waiting is visible, and destination is not.

I started with the keyboard, not the network panel, because the complaint was focus loss. Tab after the overlay appeared cycled through chrome around the chat, then died against aria-hidden on the main landmark. The overlay had been marked aria-busy="true" on #app, which is a blunt instrument and a familiar false friend.

Was the fetch still using the local URL? DevTools said no. The retry wrapper swapped the endpoint after a single 503, then streamed as if nothing privacy-related had happened. The visual skeleton was honest about waiting and dishonest about where the bytes went.

Root cause, stacked in three mistakes I now look for first:

isStreaming, represented local work, remote work, and the gap between them.tabIndex={-1} plus an effect that called overlayRef.current?.focus(). None of those is an ARIA trivia problem. They are state-modeling problems that accessibility tools merely made visible. If you “fix” them by adding more live regions without an origin enum, you will drown the one sentence that mattered.

I now keep a three-column log while reproducing AI chat failures. It is slower than jumping into CSS, and it stops me from treating a privacy change like a spinner ticket.

If column two changes host while column three stays frozen, you have a silent origin bug. If column one and the accessibility tree disagree about focus, you have a trap. Write the log before you add another aria-* attribute, because attributes cannot recover a missing phase.

I also freeze the retry in a reducer so I can replay it without waiting on a real model. The sketch below does that with a local failure, then an optional remote path. Think of origin like a shipping label on a package you already started wrapping: the packing motion can look identical, but the destination is what people must hear.

A typed union forces every view to answer a rude question: where is this request going? Automatic failover is convenient for developers and hostile for people who cannot see the URL bar change.

type Origin = "local" | "remote";

type ChatPhase =
  | { status: "idle" }
  | { status: "streaming"; origin: Origin }
  | { status: "needs-consent"; to: Origin; prompt: string }
  | { status: "error"; message: string; origin: Origin }
  | { status: "cancelled"; origin: Origin };

The needs-consent phase is the missing beat. Ask before the first remote token, keep that question in the tab order, then restore focus. I am not using a focus-trapped modal here on purpose: the last time I shipped a blocking dialog for a different chat prompt, testers lost the transcript. This banner stays in document flow.

stateDiagram-v2
  [*] --> idle
  idle --> streamingLocal: submit
  streamingLocal --> idle: done
  streamingLocal --> cancelled: stop
  streamingLocal --> needsConsent: local 503
  needsConsent --> streamingRemote: confirm
  needsConsent --> cancelled: decline
  streamingRemote --> idle: done
  streamingRemote --> cancelled: stop
  cancelled --> idle: type again

The following React sketch is a labeled demo, not a production privacy platform. It keeps the origin banner in tab order and returns focus to the composer after confirm or cancel. Screen-reader copy is short on purpose so the live region does not flood.

I needed a remote target that did not require a private GPU story in this article. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I pointed the failover branch at MonkeyCode’s free model access and free server option so the remote origin was real enough to test announcements, without standing up my own inference box.

import { useEffect, useReducer, useRef } from "react";

type Origin = "local" | "remote";

type Phase =
  | { status: "idle" }
  | { status: "streaming"; origin: Origin }
  | { status: "needs-consent"; prompt: string }
  | { status: "error"; message: string }
  | { status: "cancelled" };

type State = {
  phase: Phase;
  draft: string;
  answer: string;
  log: string;
};

type Action =
  | { type: "type"; value: string }
  | { type: "submit" }
  | { type: "local-fail"; prompt: string }
  | { type: "consent-yes" }
  | { type: "consent-no" }
  | { type: "token"; chunk: string }
  | { type: "done" }
  | { type: "cancel" };

function reducer(state: State, action: Action): State {
  switch (action.type) {
    case "type":
      return { ...state, draft: action.value };
    case "submit":
      if (state.phase.status === "streaming") return state;
      return {
        ...state,
        answer: "",
        phase: { status: "streaming", origin: "local" },
        log: "Answer streaming on this device.",
      };
    case "local-fail":
      return {
        ...state,
        phase: { status: "needs-consent", prompt: action.prompt },
        log: "Local path failed. Confirm before sending to a remote server.",
      };
    case "consent-yes":
      return {
        ...state,
        phase: { status: "streaming", origin: "remote" },
        log: "Answer streaming from a remote server.",
      };
    case "consent-no":
      return {
        ...state,
        phase: { status: "cancelled" },
        log: "Remote send cancelled. Composer is ready.",
      };
    case "token":
      return { ...state, answer: state.answer + action.chunk };
    case "done":
      return { ...state, phase: { status: "idle" }, log: "Answer complete." };
    case "cancel":
      return {
        ...state,
        phase: { status: "cancelled" },
        log: "Request cancelled.",
      };
    default:
      return state;
  }
}

export function OriginAwareChat() {
  const [state, dispatch] = useReducer(reducer, {
    phase: { status: "idle" },
    draft: "",
    answer: "",
    log: "Chat idle on this device.",
  });
  const composerRef = useRef<HTMLTextAreaElement>(null);
  const confirmRef = useRef<HTMLButtonElement>(null);
  const abortRef = useRef<AbortController | null>(null);

  useEffect(() => {
    if (state.phase.status === "needs-consent") {
      confirmRef.current?.focus();
      return;
    }
    if (
      state.phase.status === "idle" ||
      state.phase.status === "cancelled" ||
      state.phase.status === "error"
    ) {
      composerRef.current?.focus();
    }
  }, [state.phase.status]);

  async function send(origin: Origin, prompt: string, signal: AbortSignal) {
    const response = await fetch(
      origin === "local" ? "/local-infer" : "/remote-infer",
      {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ prompt }),
        signal,
      }
    );
    if (!response.ok || !response.body) throw new Error("infer-failed");
    const reader = response.body.getReader();
    const decoder = new TextDecoder();
    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      dispatch({ type: "token", chunk: decoder.decode(value) });
    }
  }

  async function onSubmit(event: React.FormEvent) {
    event.preventDefault();
    const prompt = state.draft.trim();
    if (!prompt) return;
    dispatch({ type: "submit" });
    abortRef.current?.abort();
    const controller = new AbortController();
    abortRef.current = controller;
    try {
      await send("local", prompt, controller.signal);
      dispatch({ type: "done" });
    } catch {
      if (controller.signal.aborted) dispatch({ type: "cancel" });
      else dispatch({ type: "local-fail", prompt });
    }
  }

  async function confirmRemote() {
    if (state.phase.status !== "needs-consent") return;
    const prompt = state.phase.prompt;
    dispatch({ type: "consent-yes" });
    const controller = new AbortController();
    abortRef.current = controller;
    try {
      await send("remote", prompt, controller.signal);
      dispatch({ type: "done" });
    } catch {
      if (controller.signal.aborted) dispatch({ type: "cancel" });
      else dispatch({ type: "done" });
    }
  }

  const blocked =
    state.phase.status === "streaming" ||
    state.phase.status === "needs-consent";

  return (
    <div className="chat">
      <p aria-live="polite" aria-atomic="true" className="sr-only">
        {state.log}
      </p>

      {state.phase.status === "streaming" && (
        <p className="origin-banner" tabIndex={0}>
          Origin:{" "}
          {state.phase.origin === "local" ? "this device" : "remote server"}
        </p>
      )}

      {state.phase.status === "needs-consent" && (
        <div
          className="origin-banner"
          role="region"
          aria-labelledby="consent-title"
        >
          <h2 id="consent-title">Send this prompt to a remote server?</h2>
          <p>The local path failed. A remote model will see the prompt text.</p>
          <button ref={confirmRef} type="button" onClick={confirmRemote}>
            Send remotely
          </button>
          <button type="button" onClick={() => dispatch({ type: "consent-no" })}>
            Stay on this device
          </button>
        </div>
      )}

      <div aria-live="polite">{state.answer}</div>

      <form onSubmit={onSubmit}>
        <label htmlFor="composer">Message</label>
        <textarea
          id="composer"
          ref={composerRef}
          value={state.draft}
          disabled={blocked}
          onChange={(event) =>
            dispatch({ type: "type", value: event.target.value })
          }
        />
        <button type="submit" disabled={blocked}>
          Send
        </button>
        <button
          type="button"
          onClick={() => {
            abortRef.current?.abort();
            dispatch({ type: "cancel" });
          }}
          disabled={state.phase.status !== "streaming"}
        >
          Stop
        </button>
      </form>
    </div>
  );
}

Notice the origin banner is a tab stop while streaming, not a toast. Toasts expire, and expired copy cannot be reviewed by a screen-reader cursor. The consent region sits in document order ahead of the composer, so Tab does not wander through the chrome first.

A little CSS keeps the banner readable without becoming another overlay that steals the page.

.origin-banner {
  border: 2px solid currentColor;
  padding: 0.75rem 1rem;
  margin-bottom: 1rem;
}

.origin-banner:focus {
  outline: 3px solid currentColor;
  outline-offset: 2px;
}

.sr-only {
  position: absolute;
  width: 1px;
  height: 1px;
  overflow: hidden;
  clip: rect(0, 0, 0, 0);
}

Do not put aria-busy on the document root during failover. Busy on #app hides the consent controls you just rendered, which is how I created the original trap.

aria-hidden. If any of those states reuse a single “” string, you are back to the original bug. Would you ship a checkout that retried against another processor without naming it? Then do not ship chat that way either.

Run these as regressions, not as a one-off demo day. I write the exact transition into the bug title so the next failure is comparable.

Do not announce every token. Token-level live regions bury the origin change under a speech queue nobody can interrupt cleanly. If your reader is still speaking fragments when consent appears, the user will confirm a prompt they never heard.

I am not claiming conformance from this matrix. I am claiming a reproducible path you can fail on purpose, with browser, OS, and AT versions recorded beside the transition.

Invite a teammate to reproduce with their versions plus the exact transition that failed. “It works on my laptop” is not a QA record when the bug is a silent host change.

This pattern is for product teams who already stream chat in a browser and who sometimes leave the device. It is not a privacy policy, a DPA, or a substitute for documenting retention on the remote host. The sketch also skips auth, rate limits, and model catalogs on purpose.

Do not use this approach if your product never leaves the device. Do not use it if legal consent must be a separately captured workflow with audit storage. I am not attaching quotas, hardware, duration, or benchmark numbers to the free server path, because those claims would be invented here.

The accessibility fix is the state machine, the tab-stop banner, and the focus return. A reachable remote origin only made the failing transition cheaper to replay. If you want that replay without standing up your own box, try the free model access and free server option against the consent states above, and log the AT combo that still misses the origin change.