{"slug": "updates-to-ustp-secure", "title": "Updates to USTP-Secure", "summary": "USTP-Secure (USTPS), a UDP-based secure transport protocol, has entered beta with authenticated packet protection, optional congestion control, and support for multiple AEAD ciphers. The protocol uses X25519 key exchange, per-client session keys, and TOFU for server key verification, targeting streaming applications over unordered UDP delivery.", "body_md": "USTPS means **UDP Speedy Transmission Protocol Secure**.\n\nUSTP-Secure keeps USTP on UDP and adds authenticated packet protection.\n\nBy default, USTPS uses AEAD for `DATA`\n\n.\n\nIt also supports an optional negotiated `cleartext + HMAC`\n\nmode for `DATA`\n\n, where payload bytes are visible on the wire but tampering is detected and rejected.\n\nUSTPS now supports an optional congestion controller called `USTPS Congestion`\n\n. It is still UDP-first and still keeps unordered delivery, but it can optionally slow down and ramp back up when the path starts showing congestion signals.\n\nStatus: **Beta**\n\nUSTPS is no longer just a proof of concept. It is currently in the Beta phase.\n\nUSTPS can be used for many kinds of applications and transports.\n\nThis repository, however, is focused specifically on **streaming over USTPS**.\n\n- Built with\n`Codex`\n\nusing`GPT-5.4 (Low)`\n\n. - Verified without freezing at\n`--loss 33`\n\n. - Test path:\n`Brazil -> Canada`\n\nwith about`140ms`\n\nRTT.\n\n- Transport remains UDP (no TCP tunnel)\n- AEAD ciphers:\n`chacha20`\n\n=`CHACHA20_POLY1305`\n\n`aes-256-gcm`\n\n=`AES_256_GCM`\n\n`aes-128-gcm`\n\n=`AES_128_GCM`\n\n- Default AEAD cipher:\n`chacha20`\n\n- Default\n`DATA`\n\nprotection mode is AEAD. - Optional\n`DATA`\n\nprotection mode is cleartext + per-packet HMAC. - In cleartext mode, payload bytes are not encrypted, but modifications are detected and invalid packets are discarded.\n- Transport control packets (\n`HELLO`\n\n,`ACK`\n\n,`RETRANSMIT_REQUEST`\n\n,`CLOSE`\n\n) stay plaintext on purpose. - Control packets are serialized as ASCII transport records.\n`ACK`\n\nand`NACK`\n\n/`RETRANSMIT_REQUEST`\n\nremain plaintext, but are authenticated with a per-session HMAC tag.- This prevents off-path forged ACK/NACK control packets from forcing ACK attacks or retransmission DoS after the secure session is established.\n`DATA`\n\npackets use a binary frame format named`UPACK`\n\n(`UPAK`\n\non the wire).- No static PSK is used.\n- Each client performs an X25519 key exchange when it joins.\n- Each client gets a separate AEAD session key.\n- Servers support multiple clients.\n- The server validates the client with a challenge round-trip on the source\n`IP:port`\n\n. - If\n`--cipher`\n\nis set on the server, the server uses that exact cipher. - If\n`--cipher`\n\nis omitted or set to`auto`\n\n, the server uses the cipher requested by the client. - Clients reject unexpected cipher negotiation.\n`DATA`\n\nprotection mode is negotiated separately from cipher choice:- server:\n`--cleartext auto|on|off`\n\n- client:\n`--cleartext on|off`\n\n- server default:\n`auto`\n\n- client default:\n`off`\n\n- server:\n- With server\n`auto`\n\n, the server follows the client request. - With server\n`on`\n\n, the server forces cleartext + HMAC. - With server\n`off`\n\n, the server forces AEAD. - TOFU (Trust On First Use) is enabled on the client to detect unexpected server key changes after the first connection.\n- The server keeps a persistent X25519 host key in\n`~/.ustps_host_key`\n\nby default so TOFU remains stable across reconnects and restarts. - A normal server restart does not change the host key.\n- Use\n`--regen-key`\n\non the server only when you intentionally want to rotate that host key.\n\n`ACK:`\n\n,`NACK:`\n\n,`HELLO:`\n\n, and`CLOSE:`\n\nare the plaintext control record prefixes.`USS1`\n\nmeans`UDP Speedy Secure`\n\n, version 1.`USC1`\n\nmeans`UDP Speedy Clear`\n\n, version 1.`UPAK`\n\nis the binary`UPACK`\n\nDATA frame marker.- In USTPS, plaintext control is human-readable ASCII such as\n`ACK: 10`\n\n,`NACK: 42`\n\n,`HELLO: ...`\n\n, and`CLOSE:`\n\n. - In USTPS,\n`UPAK`\n\nidentifies binary DATA packets after decryption. - In USTPS,\n`USS1`\n\nis the outer secure AEAD envelope format. - In USTPS,\n`USC1`\n\nis the outer cleartext+HMAC`DATA`\n\nenvelope format. - So, on the wire you normally see:\n`USS1...`\n\nfor AEAD-protected`DATA`\n\n`USC1...`\n\nfor cleartext+HMAC`DATA`\n\n- readable control lines for transport control\n\n- USTPS is reliable over UDP, but it is\n**unordered by design**. - USTPS can run with optional\n`USTPS Congestion`\n\n, negotiated during the handshake. - Packets carry both a transport\n`seq`\n\nand an application-facing`stream_pos`\n\n. `seq`\n\nis used for ACK, loss detection, retransmission, and`RTT`\n\nsampling.`stream_pos`\n\ntells the application where the payload belongs in the logical byte stream.- In the current implementation,\n`seq`\n\nis a 32-bit counter that starts at`1`\n\nfor each fresh session. - In the current implementation,\n`stream_pos`\n\nis a 64-bit byte counter that starts at`0`\n\nfor each fresh logical stream. - The receiver accepts out-of-order packets immediately instead of blocking delivery behind one missing packet.\n\nExample:\n\n- Physical arrival:\n`1 2 3 5 6`\n\n- Packet\n`4`\n\nis missing, so the receiver buffers the gap information and sends`RETRANSMIT_REQUEST`\n\nfor`4`\n\n. - Packets\n`5`\n\nand`6`\n\nare still accepted immediately. - When packet\n`4`\n\narrives later, the application can reconstruct the logical order by using`stream_pos`\n\n, not by trusting arrival order.\n\n- The client starts with a plaintext transport\n`HELLO`\n\ncarrying its X25519 public key, requested cipher, requested congestion-control mode (`on`\n\nor`off`\n\n), and requested`DATA`\n\nprotection mode (`cleartext on|off`\n\n). - The server does not send media immediately. It first sends a plaintext challenge containing:\n- a random retry token\n- a generated Base64\n`session_id`\n\n- the selected cipher\n- the negotiated congestion-control mode\n- the negotiated\n`DATA`\n\nprotection mode - the server public key\n\n- The client must answer with that exact same token and session metadata.\n- Only after that token round-trip succeeds does the server create the session and begin sending\n`DATA`\n\n. - After validation, the session is bound to the source\n`IP:port`\n\nthat completed the challenge. `session_id`\n\nis still used as a session label, but it is not accepted from a different`IP:port`\n\n.\n\n- USTPS uses a plaintext retry-token step before any encrypted media session is accepted.\n- Flow:\n- client sends\n`HELLO`\n\n- server replies with\n`USTPS-CHALLENGE1`\n\ncarrying`token`\n\n,`session_id`\n\n, selected cipher, negotiated congestion-control mode, negotiated`DATA`\n\nprotection mode, and server public key - client echoes that token back in\n`USTPS-CHALLENGE-REPLY1`\n\n- only then does the server create the session and send\n`USTPS-SESSION1`\n\n- client sends\n- Purpose:\n- prove that the sender at that source\n`IP:port`\n\ncan actually receive packets there - avoid sending encrypted media immediately to an unvalidated source address\n- bind the final session creation to the endpoint that completed the round-trip\n\n- prove that the sender at that source\n- The retry token is not the session key.\n- It is only a reachability proof and handshake gate before the real USTPS session is created.\n- It is also not used as a nonce, not used as an ACK/NACK MAC key by itself, and not reused as packet payload state.\n\n- Current\n`UPACK`\n\nDATA payload limit:`1200`\n\nbytes. `UPACK`\n\nfixed header:`20`\n\nbytes.- Outer\n`USS1`\n\nsecure envelope overhead in AEAD mode:`4`\n\nbytes magic`1`\n\nbyte cipher id`12`\n\nbytes AEAD nonce`16`\n\nbytes AEAD tag\n\n- So the encrypted USTPS DATA datagram is about\n`1253`\n\nbytes before IP/UDP headers. - Outer\n`USC1`\n\ncleartext envelope overhead in cleartext mode:`4`\n\nbytes magic`16`\n\nbytes HMAC tag\n\n- So the cleartext USTPS DATA datagram is about\n`1240`\n\nbytes before IP/UDP headers. - With IPv4 + UDP headers, that is about\n`1281`\n\nbytes on the wire. - With IPv6 + UDP headers, that is about\n`1301`\n\nbytes on the wire. - USTPS currently does\n**not** implement transport-level fragmentation. - USTPS currently does\n**not** implement PMTU discovery. - The implementation instead uses a fixed conservative payload ceiling.\n- If IP fragmentation still happens underneath and one fragment is lost, the whole UDP datagram is lost and USTPS recovers it with normal selective retransmission.\n\n- Duplicate packets inside the current session are ignored after their\n`seq`\n\nwas already accepted. - Very old packets can age out of the receiver history window and then be ignored as stale.\n- Old ACK/NACK packets for data that has already been retired from the retransmission buffer are ignored.\n- A stale control packet does not recreate an already-finished packet in the sender.\n\n`DATA`\n\nencryption uses a fresh random`12`\n\n-byte AEAD nonce per encrypted packet.- The nonce is generated randomly, not derived from\n`seq`\n\n. - Nonce reuse with the same session key is forbidden.\n`seq`\n\nis for transport reliability.`stream_pos`\n\nis for logical application ordering.`nonce`\n\nis only for AEAD packet protection.- Cleartext+HMAC mode does not use an AEAD nonce because it does not use AEAD encryption for\n`DATA`\n\n.\n\n`USTPS Congestion`\n\nis optional.- It does not change USTPS into an ordered transport and it does not add TCP-style HoL blocking.\n- It only changes how aggressively the sender injects packets into the network.\n- Negotiation model:\n- server:\n`--congestion-control auto|on|off`\n\n- client:\n`--congestion-control on|off`\n\n- server:\n- Default behavior:\n- server default is\n`auto`\n\n- client default is\n`off`\n\n- with server\n`auto`\n\n, the server follows what the client asked for - with server\n`on`\n\n, congestion control is forced on even if the client asked for`off`\n\n- with server\n`off`\n\n, congestion control is forced off even if the client asked for`on`\n\n- server default is\n- Runtime behavior:\n- starts at a normal send rate\n- gradually increases the effective send window and burst size while the path stays healthy\n- watches measured\n`RTT`\n\n, retransmission timeout events (`RTO`\n\n), and explicit retransmit requests (`NACK`\n\n) - if\n`RTT`\n\ninflates,`RTO`\n\nstarts happening, or loss/retransmit pressure rises, it backs off - once the path stabilizes again, it slowly ramps back up\n\n- The sender still uses selective retransmission for missing packets only.\n`USTPS Congestion`\n\ncontrols rate pressure, not reliability semantics.\n\n- Automatic network/path migration has been removed from this implementation.\n- If the client changes network and its source\n`IP:port`\n\nchanges, the current session is expected to end and the client should reconnect cleanly. - The migration implementation was removed because it caused practical reliability and security problems:\n- repeated migration floods when NAT or mobile networks changed paths quickly\n- stale sessions that looked recovered but no longer delivered media\n- long silent periods followed by GAP-only behavior\n- ambiguity between a real roaming client and spoofed packets claiming an existing\n`session_id`\n\n- complex recovery state that could reset stream ordering or retransmission state at the wrong time\n\n- The current model is intentionally simpler: prove reachability with a challenge on the current\n`IP:port`\n\n, bind the session to that endpoint, and reconnect if the endpoint changes.\n\n`ACK`\n\nis serialized like`ACK: 10 MAC:<tag>`\n\nor batched like`ACK: 10 11 12 ... MAC:<tag>`\n\n.`RETRANSMIT_REQUEST`\n\nis serialized like`NACK: 42 MAC:<tag>`\n\nor batched like`NACK: 42 43 44 ... MAC:<tag>`\n\n.`HELLO`\n\nis serialized like`HELLO: <base64-payload>`\n\n.`CLOSE`\n\nis serialized like`CLOSE:`\n\n.`DATA`\n\nuses the binary`UPACK`\n\nframe format instead of ASCII to avoid bloating media payload packets.- In AEAD mode, captures typically look like readable control lines plus encrypted\n`USS1...`\n\ndatagrams that decrypt to`UPAK...`\n\nDATA frames. - In cleartext mode, captures typically look like readable control lines plus authenticated\n`USC1...`\n\ndatagrams carrying visible`UPACK`\n\nbytes and an HMAC tag. - The\n`MAC:<tag>`\n\nvalue is computed from the session key and is stripped after verification before the packet reaches the transport state machine.\n\n- USTPS uses selective retransmission, not Go-Back-N.\n- Every unique\n`DATA`\n\npacket is ACKed individually. - Missing packets trigger\n`RETRANSMIT_REQUEST`\n\nonly for the missing`seq`\n\n. - The sender keeps sent packets in a retransmission buffer until ACKed.\n`RTO`\n\nis not fixed-only: it is adapted from measured`RTT`\n\nsamples of non-retransmitted packets.- Only the missing packets are retransmitted.\n\n`ACK`\n\n: the receiver acknowledged one or more`seq`\n\nvalues, so the sender can retire them from the retransmission buffer.`NACK`\n\n: the receiver detected a missing`seq`\n\nand explicitly requested retransmission of that missing packet only.`GAP`\n\n: the client received a packet whose`stream_pos`\n\nis ahead of the next ordered output position, so there is currently a hole in the logical byte stream.`RECOVERY`\n\n: a late packet arrived with`stream_pos`\n\nbelow the current frontier, meaning an earlier gap is being repaired or was repaired after newer data had already been seen.`RESYNC`\n\n: the client anchored ordered output to a new`stream_pos`\n\nafter a clean stream-state reset.`RTO`\n\n: retransmission timeout. The sender did not see ACK progress in time, so it queued a packet for retry even without an explicit NACK.`no data for 10s`\n\n: the client did not receive stream data for long enough and exits so a new clean session can be started.`stream state reset`\n\n: the client cleared local reorder/gap state after a clean new stream/session boundary.\n\n- TCP has transport-level Head-of-Line blocking: if one segment is missing, later data in the same byte stream cannot be delivered to the application yet.\n- USTPS does not do that at the transport layer.\n- A missing packet does not stop later packets from being received, ACKed, buffered, or passed upward.\n- That is why USTPS can physically observe flows like\n`5, 6, 4, 7, 8`\n\nwhile still preserving enough metadata for the application to rebuild the logical order if it wants ordered output.\n\nImportant:\n\n- If your final application output is a strict ordered byte stream, then reordering still has to happen somewhere above USTPS.\n- In that case, the application layer may still choose to wait before emitting bytes, but that waiting is an application behavior, not transport-level HoL blocking inside USTPS itself.\n\n- Treat USTPS as a reliable unordered datagram transport with stream position metadata.\n- Do not assume packet arrival order is the real stream order.\n- Use\n`stream_pos`\n\nto rebuild ordered output when your application needs a byte stream. - If your application can consume unordered chunks directly, you can process payloads immediately and avoid ordered buffering entirely.\n- If your application needs ordered output, keep a reorder buffer keyed by\n`stream_pos`\n\nand release data only when the required positions are available. - Do not rebuild ordering by\n`seq`\n\n; use`seq`\n\nonly for transport reliability logic.\n\n- TCP: TCP is reliable and ordered, but that ordering is enforced by the transport itself. A single missing segment blocks later data in the same stream.\n- TCP with multiplexing above it: Even if an application multiplexes many logical channels over one TCP connection, loss in the underlying TCP byte stream still blocks progress behind the gap.\n- QUIC: QUIC removes cross-stream HoL blocking between different streams, which is a big improvement over TCP for multiplexed applications.\n- QUIC stream behavior: Inside one individual QUIC stream, ordering is still enforced. Missing data in that stream blocks later bytes for that same stream.\n- USTPS:\nUSTPS does not enforce ordered delivery at the transport layer. It accepts later packets without waiting for earlier missing ones, and relies on\n`stream_pos`\n\nmetadata if the application wants to reconstruct ordered output. If`USTPS Congestion`\n\nis enabled, the sender may slow or speed up, but that does not change the unordered transport model.\n\n```\npython3 server.py \\\n  --peer-port 0 \\\n  --bind-ip 0.0.0.0 \\\n  --bind-port 40001 \\\n  --video \"<HLS_URL_OR_LOCAL_FILE>\" \\\n  --stream-container mpegts \\\n  --cipher chacha20 \\\n  --congestion-control auto \\\n  --cleartext auto\n```\n\nThe default stream container is `mpegts`\n\nbecause it is the most reliable option with VLC in the current TCP-local playback path.\n\nYou can change the FFmpeg muxer/container with `--stream-container`\n\n.\n\nExamples:\n\n`--stream-container mpegts`\n\n(default, classic MPEG-TS compatibility)`--stream-container flv`\n\n(experimental here; VLC may misdetect it as audio-only in this pipeline)`--stream-container nut`\n\n(low overhead FFmpeg-native streaming container, but VLC may not open it)`--stream-container matroska`\n\n(MKV/Matroska)\n\nIf you want custom ffmpeg encoding/transcoding parameters instead of the default copy mode, use `--video-parameters`\n\n.\n\nExample:\n\n```\npython3 server.py \\\n  --peer-port 0 \\\n  --bind-ip 0.0.0.0 \\\n  --bind-port 40001 \\\n  --video \"<HLS_URL_OR_LOCAL_FILE>\" \\\n  --stream-container mpegts \\\n  --video-parameters \"-c:v libx264 -preset veryfast -b:v 2500k -c:a aac -b:a 128k\" \\\n  --cipher chacha20 \\\n  --cleartext off\n```\n\nBehavior:\n\n- without\n`--video-parameters`\n\n: uses`-c copy`\n\n- with\n`--stream-container mpegts`\n\nand no`--video-parameters`\n\n: also adds`-mpegts_flags +resend_headers`\n\n- with\n`--video-parameters`\n\n: uses exactly what you passed instead of the default copy settings - the selected container is always passed to FFmpeg as\n`-f <stream-container>`\n\n`--loss`\n\nsimulates outbound packet loss on the server side for testing recovery behavior.- Value range:\n`0`\n\nto`100`\n\n- Example:\n\n```\npython3 server.py \\\n  --peer-port 0 \\\n  --bind-ip 0.0.0.0 \\\n  --bind-port 40001 \\\n  --video \"<HLS_URL_OR_LOCAL_FILE>\" \\\n  --cipher chacha20 \\\n  --loss 40\n```\n\n`--loss 0`\n\nmeans no simulated loss.`--loss 40`\n\nmeans the server randomly drops about 40% of its outbound packets before they leave the process.- This is useful for validating:\n- retransmission behavior\n- gap detection\n- ACK/NACK handling\n- playback resilience under controlled packet loss\n\n- In normal real-world usage, leave\n`--loss`\n\nat`0`\n\n.\n\n```\npython3 client.py \\\n  --peer-ip <SERVER_IP_OR_DOMAIN> \\\n  --peer-port 40001 \\\n  --bind-ip 0.0.0.0 \\\n  --bind-port 0 \\\n  --output-mode tcp \\\n  --tcp-host 127.0.0.1 \\\n  --tcp-port 1238 \\\n  --cipher chacha20 \\\n  --congestion-control off \\\n  --cleartext off\n```\n\nExamples:\n\n- Request normal AEAD mode:\n`--cleartext off`\n\n- Request cleartext + HMAC mode:\n`--cleartext on`\n\nNotes:\n\n- The default playout/reorder delay is now\n`1500ms`\n\n. - The client stores the first seen server X25519 public key in\n`~/.ustps_known_hosts.json`\n\n. - If that key changes later, the client aborts with a TOFU mismatch error instead of silently trusting the new key.\n- If you intentionally rotated the server host key, run the client with\n`--regen-key`\n\nto allow replacing the stored TOFU key after interactive confirmation. - TOFU entries are stored per\n`<peer-ip-or-domain>:<peer-port>`\n\n, so a different server at a different address/port is treated as a different host identity.\n\n`--udp-unordered-live`\n\nis dangerous and generally not recommended for normal media players.- In that mode, payloads are forwarded immediately in raw arrival order.\n- If a packet is retransmitted later, a generic player may treat that recovered payload as if it were a brand-new frame or packet instead of late data that belongs earlier in the logical stream.\n- That can cause visible corruption, duplicated playback artifacts, decoder confusion, or unstable playback.\n- For normal player compatibility, prefer local\n`TCP`\n\noutput or ordered UDP output with a reorder buffer.\n\nVLC:\n\n```\ntcp://127.0.0.1:1238\n```\n\n- USTP: reliable UDP transport, no encryption by default.\n- USTPS: same UDP transport plus authenticated\n`DATA`\n\nprotection, human-readable plaintext transport control, challenge validation before data flow, and endpoint-bound sessions. - Client exits with explicit error if no valid encrypted packets are received after the handshake finishes.\n\n`USTPS`\n\nInternet-Draft:`https://datatracker.ietf.org/doc/draft-x1co-ustps/`\n\n`USSH`\n\n: a shell/remote terminal protocol implemented fully from scratch on top of USTPS:`https://github.com/x1colegal/USSH`", "url": "https://wpnews.pro/news/updates-to-ustp-secure", "canonical_source": "https://github.com/x1colegal/USTP-Secure", "published_at": "2026-06-18 18:17:41+00:00", "updated_at": "2026-06-18 18:31:54.796882+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools"], "entities": ["USTPS", "UDP", "AEAD", "X25519", "CHACHA20_POLY1305", "AES_256_GCM", "AES_128_GCM", "HMAC"], "alternates": {"html": "https://wpnews.pro/news/updates-to-ustp-secure", "markdown": "https://wpnews.pro/news/updates-to-ustp-secure.md", "text": "https://wpnews.pro/news/updates-to-ustp-secure.txt", "jsonld": "https://wpnews.pro/news/updates-to-ustp-secure.jsonld"}}