Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
e58599c
feat(resolver): offer a mapper's optional asset listing in /admin/status
includeamin Sep 30, 2026
24dc1cf
feat(demo,admin): pick the asset from a dropdown, with Custom… for an…
includeamin Sep 30, 2026
b312fd0
docs(design): add TDD 0009, common encryption and DRM
includeamin Sep 30, 2026
de62037
feat(cenc): validate the mapper's encryption keys and DRM systems
includeamin Sep 30, 2026
375306f
fix(cenc): keep the wire key out of Debug output
includeamin Sep 30, 2026
ae0dc88
feat(cenc): add the cbcs pattern cipher
includeamin Sep 30, 2026
318a1f4
feat(cenc): find where each H.264 slice header ends
includeamin Sep 30, 2026
91fa376
feat(fmp4): write protected init segments and senc/saiz/saio fragment…
includeamin Sep 30, 2026
94db202
feat(cenc): encrypt an H.264 or audio segment into a cbcs fragment
includeamin Sep 30, 2026
d89e3a4
feat(protocol): signal cbcs and DRM systems in HLS and DASH
includeamin Sep 30, 2026
c4e6f18
feat(cenc): encrypt assets the mapper supplies keys for
includeamin Sep 30, 2026
00a284c
test(resolver): check the parse error itself never echoes a key
includeamin Sep 30, 2026
9dbb473
test(cenc): FFmpeg decrypts what we serve; fuzz the H.264 slice-heade…
includeamin Sep 30, 2026
b6a0cda
docs(cenc): document encryption for mapper authors and operators
includeamin Oct 1, 2026
a6e1588
fix(cenc): hold an encrypted segment's job slot until its response ends
includeamin Oct 1, 2026
b6e9484
fix(cenc): give each EXT-X-SESSION-KEY line once for split keys
includeamin Oct 1, 2026
7af54be
fix(cenc): reject FairPlay with separate video and audio keys
includeamin Oct 1, 2026
de0aaa5
docs(cenc): re-key with a 200, and note pssh-only and size limits
includeamin Oct 1, 2026
015d342
fix(lint): fix linting errors
includeamin Oct 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ categories = ["multimedia::video", "web-programming::http-server"]
exclude = ["/.github", "/docs", "/demo", "/fuzz", "/tests/fixtures", "/Dockerfile", "/.dockerignore", "/book.toml", "/Makefile"]

[dependencies]
aes = "0.9"
axum = "0.8"
base64 = "0.23"
bytes = "1"
hyper-util = { version = "0.1", features = ["http1", "http2", "server-auto", "server-graceful", "service", "tokio"] }
reqwest = { version = "0.13.5", default-features = false, features = ["rustls", "http2"] }
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,10 @@ proportional to the segment asked for, not to the length of the video.
- **No transcoding.** The codecs must already suit the protocol, and there is no bitrate ladder
unless you supply the renditions ([mapper API](docs/mapper-api.md#renditions)). Trimming and
joining files is done on keyframes, not by re-encoding ([clips](docs/mapper-api.md#clips)).
- **No DRM, no live streaming, no MPEG-TS output.** Segments are fragmented MP4. DRM is planned
after the features above.
- **No live streaming, no MPEG-TS output.** Segments are fragmented MP4. DRM is supported through
`cbcs` encryption with Widevine, FairPlay, PlayReady, and Clear Key signalling
([mapper API](docs/mapper-api.md#encryption)); video encryption is H.264 for now, with HEVC
next.
- **No authentication.** Run it behind a reverse proxy or CDN that provides it; see
[operations](docs/operations.md). TLS itself is optional but built in ([`[server.tls]`](docs/operations.md#tls))
for the case of running with no proxy in front.
Expand All @@ -51,7 +53,7 @@ proportional to the segment asked for, not to the length of the video.
| --- | --- |
| Convert files to HLS/DASH once and host static files | FFmpeg, [Shaka Packager](https://github.com/shaka-project/shaka-packager), or Bento4 |
| Serve HLS/DASH without storing packaged output, as a standalone service, with strict resource limits | **segmentor** |
| DRM, live streaming, or MPEG-TS output today | Shaka Packager |
| Live streaming or MPEG-TS output today | Shaka Packager |
| A mature on-demand packager that runs inside nginx | Kaltura's [nginx-vod-module](https://github.com/kaltura/nginx-vod-module) (AGPL-3.0) |

segmentor takes its idea, packaging on demand instead of ahead of time, from nginx-vod-module but
Expand Down
62 changes: 42 additions & 20 deletions admin/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,10 @@ <h1>segmentor control panel</h1>
<div class="panel">
<div class="panel-head"><h2>Player</h2></div>
<div style="display:flex; gap:8px; flex-wrap:wrap; margin-bottom:8px">
<label>Known asset
<select id="asset-picker"><option value="">— pick one, or type below —</option></select>
<label>Asset
<select id="asset-pick"><option value="sample">sample</option><option value="">Custom…</option></select>
</label>
<label>Asset <input id="asset" size="14" value="sample"></label>
<label id="asset-custom" hidden>Name <input id="asset" size="14" placeholder="asset id"></label>
<label>Protocol<select id="proto"><option value="hls">HLS</option><option value="dash">DASH</option></select></label>
<button id="load" style="align-self:end">Play</button>
</div>
Expand Down Expand Up @@ -163,7 +163,7 @@ <h1>segmentor control panel</h1>
if (!diagnose()) log('<span class="warn-text">cannot start, see warnings under the player</span>');
$('log').innerHTML = '';
markCurrentRow();
const asset = encodeURIComponent($('asset').value);
const asset = encodeURIComponent(assetId());
if ($('proto').value === 'hls') {
const url = `${base()}/hls/${asset}/master.m3u8`;
log('HLS ' + url);
Expand Down Expand Up @@ -197,28 +197,45 @@ <h1>segmentor control panel</h1>
}

/// Plays a row's asset id when a cache or catalog row is clicked, so browsing needs no typing.
function playAsset(assetId) {
$('asset').value = assetId;
function playAsset(id) {
selectAsset(id);
load();
}

function markCurrentRow() {
const current = $('asset').value;
const current = assetId();
for (const row of document.querySelectorAll('#cache tbody tr'))
row.classList.toggle('current', row.dataset.asset === current);
// The picker only ever shows a known asset; typing a custom name falls back to its
// placeholder rather than pretending one of the listed assets is selected.
const picker = $('asset-picker');
picker.value = [...picker.options].some(o => o.value === current) ? current : '';
}

/// The dropdown next to the Asset field: every asset ID the cache table lists. Picking one plays
/// it; typing a name that isn't listed (a mapper asset nobody has requested yet, for example)
/// keeps working in the text field regardless, since the picker is a convenience, not a gate.
function renderAssetPicker(ids) {
$('asset-picker').innerHTML = '<option value="">— pick one, or type below —</option>'
+ ids.map(id => `<option value="${escapeHtml(id)}">${escapeHtml(id)}</option>`).join('');
markCurrentRow();
// ---------- asset dropdown ----------
// Every asset the server can name (its static catalog, or a mapper's optional listing) plus
// whatever it has loaded, then "Custom…" for any other name. Asset IDs are never empty, so the
// empty value marks "Custom…" without colliding with one.
const assetId = () => $('asset-pick').value || $('asset').value.trim();

function showCustomField() {
$('asset-custom').hidden = $('asset-pick').value !== '';
}

/// Rebuilds the options from `ids`, keeping the current choice: a selected asset stays selected
/// even when the list no longer names it, and "Custom…" stays chosen with its text intact.
function renderAssetOptions(ids) {
const pick = $('asset-pick'), selected = pick.value;
const all = [...new Set(selected ? [...ids, selected] : ids)].sort();
pick.innerHTML = all.map(id => `<option value="${escapeHtml(id)}">${escapeHtml(id)}</option>`).join('')
+ '<option value="">Custom…</option>';
pick.value = selected;
showCustomField();
}

/// Selects `id`, adding it to the dropdown if it is not listed yet.
function selectAsset(id) {
const pick = $('asset-pick');
if (![...pick.options].some(option => option.value === id))
pick.add(new Option(id, id), pick.options.length - 1);
pick.value = id;
showCustomField();
}

function renderPlayerStats() {
Expand Down Expand Up @@ -349,7 +366,8 @@ <h1>segmentor control panel</h1>
: '';
for (const row of $('cache').tBodies[0].rows)
row.onclick = () => playAsset(row.dataset.asset);
renderAssetPicker([...rows.keys()].sort());
renderAssetOptions([...rows.keys()]);
markCurrentRow();
}

function escapeHtml(s) {
Expand Down Expand Up @@ -406,7 +424,11 @@ <h1>segmentor control panel</h1>

$('load').onclick = load;
$('asset').addEventListener('input', markCurrentRow);
$('asset-picker').addEventListener('change', () => { if ($('asset-picker').value) playAsset($('asset-picker').value); });
$('asset').addEventListener('keydown', event => { if (event.key === 'Enter') load(); });
$('asset-pick').addEventListener('change', () => {
showCustomField();
if ($('asset-pick').value) load(); else $('asset').focus();
});
load();
clearTimeout(pollTimer);
poll();
Expand Down
68 changes: 46 additions & 22 deletions demo/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -31,24 +31,23 @@
#log { height:150px; overflow:auto; font:11px/1.5 ui-monospace,monospace; color:var(--dim); }
#status { font-size:12px; color:var(--dim); }
.ok { color:var(--ok); } .bad { color:var(--bad); } .warn { color:var(--warn); }
.chip { background:var(--panel); border:1px solid var(--line); border-radius:99px; padding:2px 10px; color:var(--fg); cursor:pointer; font:inherit; font-size:12px; }
.chip:hover { border-color:var(--acc); color:var(--acc); }
.chip.current { border-color:var(--acc); color:var(--acc); background:rgba(76,194,255,.12); }
</style>
</head>
<body>
<header>
<h1>segmentor demo</h1>
<label>Server <input id="base" size="26" value="http://127.0.0.1:3000"></label>
<label>Asset <input id="asset" size="12" value="sample"></label>
<label>Asset
<select id="asset-pick"><option value="sample">sample</option><option value="">Custom…</option></select>
</label>
<label id="asset-custom" hidden>Name <input id="asset" size="14" placeholder="asset id"></label>
<label>Protocol
<select id="proto"><option value="hls">HLS</option><option value="dash">DASH</option></select>
</label>
<label>Poll (s) <input id="poll" type="number" min="0.5" step="0.5" value="1" style="width:70px"></label>
<button id="load">Load</button>
<span id="status"></span>
</header>
<div id="try-row" style="padding:0 16px 10px;font-size:12px;color:var(--dim)"></div>

<main>
<div class="stack">
Expand Down Expand Up @@ -106,23 +105,45 @@ <h2>Resolver &amp; cache</h2>
const tile = (k, v, cls='') => `<div class="tile"><div class="k">${k}</div><div class="v ${cls}">${v}</div></div>`;
const escapeHtml = s => String(s).replace(/[&<>"']/g, c => ({ '&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;' }[c]));

// ---------- assets to try ----------
// A quick way in besides typing a name: whatever the static catalog lists (if any) plus whatever
// has already been played (any resolver). /admin/status has the same trust model as /metrics,
// which this page already reads, so nothing new is exposed by also reading this.
async function refreshTryRow() {
// ---------- asset dropdown ----------
// Every asset the server can name (its static catalog, or a mapper's optional listing) plus
// whatever it has loaded, then "Custom…" for any other name. Asset IDs are never empty, so the
// empty value marks "Custom…" without colliding with one.
const assetId = () => $('asset-pick').value || $('asset').value.trim();

function showCustomField() {
$('asset-custom').hidden = $('asset-pick').value !== '';
}

/// Rebuilds the options from `ids`, keeping the current choice: a selected asset stays selected
/// even when the list no longer names it, and "Custom…" stays chosen with its text intact.
function renderAssetOptions(ids) {
const pick = $('asset-pick'), selected = pick.value;
const all = [...new Set(selected ? [...ids, selected] : ids)].sort();
pick.innerHTML = all.map(id => `<option value="${escapeHtml(id)}">${escapeHtml(id)}</option>`).join('')
+ '<option value="">Custom…</option>';
pick.value = selected;
showCustomField();
}

/// Selects `id`, adding it to the dropdown if it is not listed yet.
function selectAsset(id) {
const pick = $('asset-pick');
if (![...pick.options].some(option => option.value === id))
pick.add(new Option(id, id), pick.options.length - 1);
pick.value = id;
showCustomField();
}

// /admin/status has the same trust model as /metrics, which this page already reads, so nothing
// new is exposed by also reading it.
async function refreshAssets() {
try {
const status = await (await fetch(base() + '/admin/status')).json();
const ids = [...new Set([...(status.resolver.known_assets || []), ...status.cache.assets.map(a => a.asset_id)])].sort();
const row = $('try-row');
if (!ids.length) { row.textContent = ''; return; }
const current = $('asset').value;
row.innerHTML = 'Try: ' + ids.map(id =>
`<button class="chip${id === current ? ' current' : ''}" data-asset="${escapeHtml(id)}">${escapeHtml(id)}</button>`).join(' ');
for (const chip of row.querySelectorAll('.chip')) chip.onclick = () => { $('asset').value = chip.dataset.asset; load(); };
} catch { /* the try row is a convenience; leave it as it was if the origin is unreachable */ }
renderAssetOptions([...(status.resolver.known_assets || []), ...status.cache.assets.map(a => a.asset_id)]);
} catch { /* the list is a convenience; keep it as it was if the origin is unreachable */ }
}
setInterval(refreshTryRow, 10000);
setInterval(refreshAssets, 10000);

// ---------- player ----------
function teardown() {
Expand All @@ -149,7 +170,7 @@ <h2>Resolver &amp; cache</h2>
teardown();
if (!diagnose()) log('<span class="bad">cannot start, see warnings under the player</span>');
$('log').innerHTML = '';
const asset = encodeURIComponent($('asset').value);
const asset = encodeURIComponent(assetId());
if ($('proto').value === 'hls') {
const url = `${base()}/hls/${asset}/master.m3u8`;
log('HLS ' + url);
Expand Down Expand Up @@ -300,8 +321,11 @@ <h2>Resolver &amp; cache</h2>
pollTimer = setTimeout(poll, interval * 1000);
}

$('load').onclick = () => { load(); clearTimeout(pollTimer); prev = null; poll(); setTimeout(refreshTryRow, 1200); };
load(); poll(); refreshTryRow();
function start() { load(); clearTimeout(pollTimer); prev = null; poll(); setTimeout(refreshAssets, 1200); }
$('load').onclick = start;
$('asset-pick').addEventListener('change', () => { showCustomField(); if ($('asset-pick').value) start(); else $('asset').focus(); });
$('asset').addEventListener('keydown', event => { if (event.key === 'Enter') start(); });
load(); poll(); refreshAssets();
</script>
</body>
</html>
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
- [Trick play, subtitles, and adaptive renditions](technical-design/0006-trick-play-subtitles-and-renditions.md)
- [Viewer authorization](technical-design/0007-viewer-authorization.md)
- [Clipping and concatenation](technical-design/0008-clipping-and-concatenation.md)
- [Common encryption and DRM](technical-design/0009-common-encryption-and-drm.md)
- [Technical design template](technical-design/template.md)

# Implementation
Expand Down
8 changes: 4 additions & 4 deletions docs/internals/http-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,22 +94,22 @@ The most involved handler:
1. Look up asset, require `?v=`, parse the track, and answer `304` if the ETag matches.
2. **Prepare** the segment on the blocking pool (`prepare_media_segment`): this yields the header bytes, source ranges, and total length without reading payload.
3. Compute the requested byte interval from `Range` and `If-Range`; `416` if unsatisfiable.
4. If the method is `HEAD`, return headers with an empty body. No job slot, no task.
5. Otherwise acquire the first job slot (`503` on queue timeout), spawn a `StreamJob`, and return `Body::from_stream` over a two-item channel.
4. If the method is `HEAD`, return headers with an empty body. No task; a clear segment takes no job slot (an encrypted one releases the slot it was encrypted under).
5. Otherwise acquire the first job slot (`503` on queue timeout), or for an encrypted segment reuse the slot it was encrypted under, spawn a `StreamJob`, and return `Body::from_stream` over a two-item channel.

The response's `Content-Length` and `Content-Range` are known up front because the header length and payload length are known from metadata.

### `StreamJob`

An async task that produces the body. Its `stream` method:

1. Sends the part of the header (`moof` plus `mdat` header) that overlaps the requested interval.
1. Sends the part of the header (`moof` plus `mdat` header, or a whole encrypted fragment) that overlaps the requested interval, in chunks of `stream_chunk_bytes`.
2. Walks the prepared source ranges. Byte positions in the response are "virtual offsets" (header first, then each range in order); for each range it intersects the requested interval and reads that overlap in chunks of `stream_chunk_bytes`.
3. For each chunk: `read` acquires a job slot (the first read reuses the slot acquired by the handler), runs `PackagedAsset::read_range` on the blocking pool, **releases the slot**, then `send` pushes the chunk into the channel under `response_idle_timeout_ms`.

Failure handling: a source-read or permit failure sends one generic `io::Error` into the body (so the connection aborts instead of ending cleanly short) and counts an `error` abort. A full channel for longer than the idle timeout drops the stream and counts `idle`. A closed channel means the client left and counts `client`.

Invariants worth preserving: a slot is never held while waiting on the client; every send is time-bounded; nothing here reads more than the requested interval.
Invariants worth preserving: for clear segments, a slot is never held while waiting on the client (an encrypted segment's slot covers its bytes in memory until the stream ends); every send is time-bounded; nothing here reads more than the requested interval.

## Conditional and range helpers

Expand Down
Loading
Loading