Skip to content

perf(worker): match static assets on an extension allowlist - #5692

Open
vfanucci wants to merge 2 commits into
mainfrom
seo/worker-static-asset-allowlist
Open

vfanucci wants to merge 2 commits into
mainfrom
seo/worker-static-asset-allowlist

Conversation

@vfanucci

Copy link
Copy Markdown
Contributor

The worker treated any path ending in .something as a static asset and skipped the security/CSP middlewares and the edge cache for it. Plugin element pages end with a fully qualified class name (/plugins/core/flow/io.kestra.plugin.core.flow.subflow) and release pages with a version (/docs/changelog/v1.3.39), so they took that shortcut too.

Observed on the 16 Sept crawl and confirmed live today:

  • 1 947 plugin element pages + 38 changelog/migration pages served without content-security-policy, x-frame-options and the other headers every other page gets.
  • The same plugin pages never reach the edge cache: full Astro render on every hit, TTFB 0.8–10 s cold, vs 40–90 ms cf-cache-status: HIT on the plugin group pages next to them. Screaming Frog flagged 510 pages above 1.5 s, all in this set.

Static assets are now matched on an allowlist of the extensions the site actually ships (src/middlewares/staticAssets.ts, unit-tested); .md variants keep bypassing the cache as before. Also adds ke (emailing token) to the tracking params stripped from the cache key.

The worker short-circuited any path ending in `.something` to the asset
handler, skipping the security/CSP middlewares and the edge cache. Plugin
element pages end with a fully qualified class name
(`/plugins/core/flow/io.kestra.plugin.core.flow.subflow`) and release pages
with a version (`/docs/changelog/v1.3.39`), so 1 947 plugin pages and 38
docs pages were served without Content-Security-Policy / X-Frame-Options and
fully re-rendered on every request (TTFB 0.8–10 s cold vs 40–90 ms for the
cached plugin group pages).

Static assets are now recognised on an allowlist of the extensions the site
actually ships; `.md` variants keep bypassing the cache as before. Also adds
`ke` (emailing token) to the tracking params stripped from the cache key.
@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

☁️ Cloudflare Worker Preview Deployed!

🔗 https://ks-seo-worker-static-asset-allowl-docs.kestra-io.workers.dev
🔗 https://a3d4b1e7-docs.kestra-io.workers.dev

🔦 Lighthouse Benchmark

Tested on 2026-09-22 08:58 UTC  ·  links point to https://ks-seo-worker-static-asset-allowl-docs.kestra-io.workers.dev
Compared against main baseline from 2026-09-22
Runner CPU index per shard: 2487, 4139, 3139, 2487 (baseline: 2486, deltas hidden on 11 page(s))

Scores (0–100, higher is better)

Page Performance Accessibility Best Practices SEO
Home 91 84 77 92
Get Started 80 83 77 100
Pricing 99 92 77 100
Enterprise 96 83 77 100
Cloud 94 81 77 100
Features 98 92 77 92
About Us 98 91 77 100
Docs Landing 85 88 77 92
Contribute to Kestra (simple docs) 98 88 77 92
Flow (full featured docs) 92 90 77 92
Blog Index 99 91 77 100
Blog Post (sample) 96 88 77 100
VS Page (sample) 98 88 77 100
Plugins Landing 91 81 77 92
Plugin Page (sample) 93 88 77 100
Plugin Debug Page (sample) 86 88 77 100
Plugin Debug Return Page (sample) 61 88 77 100
Blueprints Landing 84 91 77 100
Blueprint Audit Logs CSV Export 75 92 77 100

Core Web Vitals (lower is better)

Page LCP FCP TBT CLS Speed Index
Home 1.44 s 0.66 s 139 ms 0.000 1.56 s
Get Started 1.52 s 0.66 s 331 ms 🔻 0.000 1.02 s
Pricing 0.88 s 0.63 s 4 ms 0.000 1.06 s
Enterprise 1.25 s 0.64 s 82 ms 0.003 1.11 s
Cloud 1.54 s 0.53 s 5 ms 0.049 1.01 s
Features 1.00 s 0.64 s 79 ms 0.000 1.20 s
About Us 1.13 s 0.58 s 39 ms 0.000 0.97 s
Docs Landing 1.50 s 1.30 s 25 ms 0.001 2.80 s
Contribute to Kestra (simple docs) 0.98 s 0.47 s 77 ms 0.000 1.04 s
Flow (full featured docs) 1.04 s 0.60 s 188 ms 0.002 1.22 s
Blog Index 0.95 s 0.57 s 56 ms 0.001 0.84 s
Blog Post (sample) 1.35 s 0.53 s 0 ms 0.000 1.06 s
VS Page (sample) 0.98 s 0.63 s 37 ms 0.000 1.05 s
Plugins Landing 1.68 s 0.69 s 45 ms 0.000 1.56 s
Plugin Page (sample) 0.81 s 0.58 s 203 ms 0.000 1.08 s
Plugin Debug Page (sample) 1.21 s 0.58 s 274 ms 0.000 1.07 s
Plugin Debug Return Page (sample) 1.09 s 0.80 s 1464 ms 0.000 2.75 s
Blueprints Landing 2.35 s 0.90 s 107 ms 0.000 🟢 1.40 s 🟢
Blueprint Audit Logs CSV Export 1.15 s 0.79 s 454 ms 0.000 1.77 s
Legend

🟢 improved  ·  🔻 regressed  ·  (blank) no significant change
Score threshold: ±10 pts  ·  Metric threshold: ±30% of baseline

Median of repeated runs: Home x5, About Us x3, Docs Landing x5, Flow (full featured docs) x3, Plugins Landing x3, Blueprints Landing x3. A single run of these swings 20+ points between runners.
The sample is measured across 4 parallel runners, so each page carries its own CPU index (Lighthouse's benchmarkIndex). Lighthouse does not normalise for host CPU, so a page's deltas are hidden when its runner differs from the baseline's by more than 10%.

View full Lighthouse HTML report for a page

Full per-page Lighthouse Results (LHR) are attached as the lhr-reports-shard-* artifacts on this run. Download and unzip one, then open https://googlechrome.github.io/lighthouse/viewer/ and drop the <page>-lhr.json file into the page to see every audit, opportunity, and diagnostic.

@iitzIrFan iitzIrFan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Structurally this fits the codebase: the predicate lives next to the other worker helpers in src/middlewares/ with a colocated vitest file, the same way contentSecurityPolicy.ts / contentSecurityPolicy.test.ts do; no new deps, and the comment density matches the rest of worker.ts. Tests pass (28) and lint/typecheck are clean on the branch. The diagnosis is right too — I reproduced it live: plugin element pages and /docs/changelog/v* come back with no content-security-policy today.

One gap before this lands, in the same class of bug the PR is fixing.

Two live, indexed element pages end with a segment that is an allowlisted extension, so they stay on the bypass path after this change:

  • /plugins/plugin-transform-records/io.kestra.plugin.transform.mapmap
  • /plugins/plugin-transform-records/io.kestra.plugin.transform.zipzip

Both return 200 text/html with no CSP and no cf-cache-status, and both are in sitemap/plugins.xml. I checked the full sitemap (2 613 URLs) and the plugin API (2 169 classes, 583 packages) — these two are the only collisions today, but the shape is open-ended: any future class named Csv, Json, Xml, Pdf, Wav… lands in the same hole silently.

zip can't just be dropped from the list (public/kestra-logo-kit.zip is real). map can — there's no sourcemap setting in astro.config, and no .map files are emitted — so removing it costs nothing and fixes half the case. For the rest, a narrow guard in isStaticAssetPath keeps the allowlist honest:

// Plugin element pages end with a fully qualified class name, so their last
// segment can itself end in a real extension
// (io.kestra.plugin.transform.map, …transform.zip). Only the `.md` variant
// of those pages is an asset.
const PLUGIN_ELEMENT_SEGMENT = /(^|\/)io\.kestra\.[^/]*$/

export function isStaticAssetPath(pathname: string): boolean {
    if (PLUGIN_ELEMENT_SEGMENT.test(pathname) && !pathname.endsWith(".md")) {
        return false
    }
    // … existing extension check
}

Worth two more rows in the "treats %s as a page" table so it can't regress.

Smaller notes, non-blocking:

  • The ke tracking param is unrelated to the allowlist. Fine by me to keep it here given the size, but it deserves a line in the PR title/description so it's findable later.
  • The STATIC_ASSET_EXTENSIONS comment says "extensions of the files this site actually ships", but roughly a third of the entries (mjs, cjs, wasm, avif, jpeg, ttf, otf, eot, webm, mp3, ogg, wav, csv) aren't shipped today. Not harmful for assets, but each one is a fresh collision surface for the page paths above — either trim to what's real or reword the comment to "extensions we're willing to treat as assets".

@iitzIrFan iitzIrFan moved this from To review to On hold in Pull Requests Sep 22, 2026

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: On hold

Development

Successfully merging this pull request may close these issues.

2 participants