High-performance Rust HTTP virus scanner (ClamAV CVD)

[eldadfux]

Native Rust HTTP virus scanner that parses ClamAV public CVD/CLD databases (no libclamav), streams file and hash uploads, and hot-reloads verified signature updates with zero downtime.

Playground

TanStack Start + shadcn UI in a second Docker container, wired through docker-compose.yml:

docker compose up --build

• UI: http://127.0.0.1:3000 — scan files, hash lookups, batch hashes
• API: http://127.0.0.1:8080
• Responses are parsed into verdicts, signature names, sizes, hashes, and scan time (duration_us)
• Same-origin /api/* proxies to defender (DEFENDER_URL)

CI

GitHub Actions matching other Appwrite Docker image repos (console, assistant):

• Tests — cargo test --locked plus playground npm test / typecheck
• Build — multi-arch (linux/amd64, linux/arm64) Docker build on PRs/main, push: false
• Release — publishing a GitHub Release logs into Docker Hub with vars.DOCKERHUB_USERNAME / secrets.DOCKERHUB_TOKEN and pushes appwrite/defender with semver tags (x.y.z, x.y, x)

Virus database sizes (18 Aug 2026)

File	On disk	Unpacked	Header signatures
main.cvd	84.95 MiB (89,072,577 B)	225.39 MiB	3,287,027
daily.cvd	22.34 MiB (23,426,416 B)	82.14 MiB	355,605
bytecode.cvd	0.27 MiB (281,702 B)	1.24 MiB	80 (not executed)
Total baked (main+daily)	107.56 MiB	307.5 MiB

Loaded engine (main+daily, PUA off): ~540k file hashes, ~102k body, ~307k logical. RSS ≈ 1.4 GiB.

HTTP e2e benchmarks (live TCP, Criterion, 4× x86_64)

Synthetic engine:

Endpoint	Time	Throughput
GET /health	36 µs	27.7 k req/s
POST /scan EICAR	45 µs	22.3 k req/s
POST /scan/hash	45 µs	22.4 k req/s
POST /scan clean 64 KiB	241 µs	259 MiB/s
POST /scan clean 1 MiB	3.37 ms	296 MiB/s
EICAR ×32 concurrent	583 µs/batch	54.9 k req/s

Official daily.cvd over HTTP: EICAR 44.5 µs; hash lookup 43.6 µs; clean 64 KiB 667 µs (93.7 MiB/s); EICAR ×16 concurrent 54.0 k req/s.

What it does

• Parses the 512-byte ClamAV-VDB header, gzip+tar body, MD5 integrity, and official RSA cli_versig
• Loads .hdb/.hsb (Hash:Size:Name), .mdb/.msb (Size:Hash:Name), .ndb, .ldb, FP/ignore
• Streamable POST /scan, POST /scan/hash, POST /scan/hashes
• Background updater verifies CVDs and arc-swaps the engine with zero downtime
• Docker bakes current official DBs at build time

  

[greptile-apps]

Greptile Summary

The PR adds a native Rust HTTP virus scanner, CVD parsing and updates, container/release automation, and a browser playground. The target-filtering fix closes the reported unknown-file false-positive path but makes several supported target-specific signature classes unreachable.

• Implements streaming file and hash scanning backed by parsed ClamAV databases
• Adds verified database loading, background updates, and atomic engine replacement
• Adds Docker, CI/release workflows, tests, benchmarks, and a TanStack playground
• Tightens NDB/LDB target filtering

Confidence Score: 4/5

The PR is not yet safe to merge because target-specific signatures for several supported formats are now skipped unconditionally.

The strict target predicate receives Any for HTML, OLE2, mail, text, Flash, Java, graphics, and other inputs because detect_kind cannot emit those target types, so matching NDB/LDB signatures produce false-negative scan verdicts.

Files Needing Attention: src/engine/mod.rs and src/signatures/ndb.rs

Important Files Changed

Filename	Overview
src/engine/mod.rs	Implements core scanning and target filtering, but strict equality suppresses signatures for every target type that detect_kind cannot identify.
src/signatures/ndb.rs	Parses the complete target-type range, exposing the mismatch with the engine's narrower file classifier.
src/signatures/ldb.rs	Parses logical-signature targets that are consistently guarded by the engine's target predicate.
src/http/mod.rs	Adds the streaming HTTP scan and hash endpoints with no accepted follow-up finding.
src/updater/mod.rs	Adds verified background database updates and atomic engine replacement with no accepted follow-up finding.
playground/src/lib/defender-proxy.ts	Adds the playground's same-origin Defender proxy; no follow-up-eligible finding was retained.

Prompt To Fix All With AI

### Issue 1
src/engine/mod.rs:561-563
**Target signatures become unreachable**

When an HTML, OLE2, mail, text, Flash, Java, graphics, or other file is scanned, `detect_kind` returns `Any`, so this strict equality rejects signatures targeting the file's actual format and causes matching malware to receive a clean verdict.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (2): Last reviewed commit: "Refactor target matching logic and add c..." | Re-trigger Greptile}