Skip to content

Stream Ward

Streaming AEAD · XChaCha20-Poly1305 · secretstream-style chaining

Watch a one-shot decrypt buffer a whole file into an OOM kill while a chunked stream holds flat — then truncate, reorder and drop segments against a real chained verifier, and against a naive split that never notices.

What is this?

Authenticated encryption gives you one tag — one short fingerprint — that proves a message came from someone holding the key and was not altered. To check that tag you need the whole message, which is fine for a chat line and impossible for a 10 GiB video. So real systems cut the file into segments and authenticate each one separately.

That fixes the memory problem and quietly creates a new one. Each segment is now a valid little message on its own, so an attacker who cannot read or change a single byte can still rearrange them: play chapter five before chapter three, delete the paragraph they dislike, cut the file short. Nothing was forged, so nothing fails — unless the segments were chained together in the first place.

This page shows both halves: the memory wall that forces you to chunk, and the authentication trap that chunking walks you into.

Exhibit 1

The RAM ceiling

MODELLED

One-shot AEAD cannot release a single byte of plaintext until the tag over the entire file verifies. So the entire ciphertext has to be resident first, and then the plaintext buffer is allocated on top of it. Push the file size up and watch where that ends.

Memory footprint is modelled from tracked buffer allocations, not measured process RSS. The chunk-chaining below uses real cryptographic operations.

10.0 GiB

One-shot decrypt

single tag over the whole file

Idle — press “Run both decrypts”.

Peak footprint
—
Needs to finish
—
Buffer allocations
—

Chunked stream

one tag per segment
Same trace, magnified to this panel's own peak — one segment in, one segment out

Idle — press “Run both decrypts”.

Peak footprint
—
Needs to finish
—
Segments processed
—

Both meters are drawn to the same scale, so the flat line really is that flat.

Why one-shot needs two copies of the file

A Poly1305 tag is computed over every byte of the ciphertext. Until the last byte has gone through the accumulator you do not know whether the first byte is trustworthy, so a correct one-shot implementation refuses to emit plaintext early — that refusal is the whole security property. The ciphertext therefore has to be fully resident. Decryption then writes into a separate output buffer (this is what crypto_aead_xchacha20poly1305_ietf_decrypt and WebCrypto's decrypt() both do), and for a moment both buffers are live: peak ≈ 2 × file size.

A chunked stream verifies a tag every segment, so it can hand that segment onward and free it before reading the next. Peak ≈ 2 × segment size — a number you choose, rather than one your users choose for you by picking a big file.

Exhibit 2

The chain

REAL CRYPTO

Everything below runs actual XChaCha20-Poly1305 and SHA-256 in your browser. The message is a settlement batch, cut into six segments — chosen so that moving a line changes what a bank would do, not just which bytes it received.

Construction

Sealed segments

    The construction, byte for byte 
    DOMAIN = "crypto-lab/stream-ward/v1"

chain[0]   = SHA-256( DOMAIN ‖ "chain-init" ‖ header )
nonce[i]   = SHA-256( DOMAIN ‖ "nonce" ‖ chain[i] ‖ LE64(i) )[0..24]
aad[i]     = DOMAIN ‖ "seg" ‖ LE64(i) ‖ chain[i]
inner[i]   = flag[i] ‖ plaintext[i]      flag = 0x01 on the last segment, else 0x00
ct[i]      = XChaCha20-Poly1305( key, nonce[i], aad[i] ).encrypt( inner[i] )
chain[i+1] = SHA-256( DOMAIN ‖ "chain" ‖ chain[i] ‖ LE64(i) ‖ ct[i] )

    Three things follow, and they are exactly the three attacks. chain[i] absorbs every earlier ciphertext including its Poly1305 tag, so a segment that moves lands on a chain state its tag does not cover. LE64(i) sits in the associated data, so a segment also cannot change position without changing what it was authenticated against. And the FINAL flag lives inside the encrypted block — as it does in libsodium's crypto_secretstream — so a stream that stops early ends on a segment that authenticates as MESSAGE, and the verifier can say so.

    chain[i] is not secret. It is derived from the public header and the ciphertexts, and an attacker can compute it too. Confidentiality comes from the key alone; the chain buys ordering and completeness, nothing more.

    Exhibit 3

    Break it yourself

    REAL CRYPTO

    You are now the attacker. You do not have the key, and you will not change a single byte inside any segment — you only get to move segments around or throw them away. Then the real verifier runs.

    No attack applied — the stream is exactly as it was sealed.

    What the application receives

    Exhibit 4

    Scorecard

    Six experiments. Each cell fills in with what the verifier actually reported when you ran that combination — nothing here is pre-written. 0 of 6 run.

    Attack Chained — Stream Ward Naive split

    What this does and does not prove

    Real here

    • XChaCha20-Poly1305 from @noble/ciphers, pinned by the draft-irtf-cfrg-xchacha-03 KATs.
    • SHA-256 from @noble/hashes, pinned by FIPS 180-4 KATs.
    • The chaining, framing, wire format and verifier — hand-written in this repo so you can read them.
    • Every accept and every reject on this page is the genuine verifier's answer.

    Modelled here

    • Exhibit 1 only. Memory footprint is modelled from tracked buffer allocations, not measured process RSS.
    • No file is read and no 10 GiB of data exists; the allocation ledger is walked, not filled.
    • Real allocators add fragmentation, GC pressure and copy-on-write — all of which make it worse, not better.

    Not proven, and out of scope

    • Not production crypto. This is a teaching demo. Use libsodium's crypto_secretstream_xchacha20poly1305, or an age/Tink streaming AEAD.
    • No rekeying: a real secretstream ratchets its key so one stream can outlive its nonce space.
    • No random access — chaining is strictly sequential by design. Seekable formats need a different tree.
    • Nothing about traffic analysis: segment sizes and counts leak, and this demo does not pad them.
    • The key is generated per page load, held in memory only, and never stored or transmitted.

    Glossary

    AEAD
    Authenticated Encryption with Associated Data. Encrypts a message and produces a tag proving it was not altered. "Associated data" is extra context that is authenticated but not encrypted.
    Poly1305 tag
    The 16-byte authenticator. If one bit of the ciphertext, nonce, key or associated data differs, it fails.
    Nonce
    A number used once. Reusing one with the same key is catastrophic, which is why XChaCha20's 24-byte nonce is big enough to pick at random.
    Chain state
    A 32-byte running hash of everything sealed so far. Not secret — its job is to make each segment's tag depend on its position and on all of its predecessors.
    Fail closed
    When verification fails, produce an error and no plaintext — never a "best effort" partial result.