Skip to content

Repository files navigation


ScaleDP

Process documents using AI/ML in the browser. No server, no upload.

Documentation npm License: AGPL-3.0-or-later StabRise


Source Code: https://github.com/StabRise/scaledp-ts

Python sibling: ScaleDP — the same pipeline model, on Apache Spark


scaledp-ts

@stabrise/scaledp processes PDFs and images entirely in the browser: PDF rendering, text detection, OCR and named-entity recognition, composed as a pipeline. Inference runs on onnxruntime-web over WebAssembly or WebGPU.

It mirrors the ScaleDP Python library — same stages, same parameter names, same schemas — so a pipeline reads the same in both. What differs is the runtime: no Spark, no server, and no document ever leaves the browser, which is the point for anything sensitive.

Key features

Document processing

  • Load PDFs and images from a File, Blob, ArrayBuffer or URL
  • Render PDF pages, or read an existing text layer and skip OCR entirely
  • Word-level bounding boxes throughout, so every result maps back to the page

OCR

  • PaddleOCR (PP-OCRv5/v6) via ppu-paddle-ocr — 13 language presets
  • DBNet ONNX — the same detection model ScaleDP uses server-side
  • Tesseract via tesseract-wasm, with independent script detection

NLP

  • GLiNER zero-shot NER: entity types are plain-language labels given at call time, not fixed by the model
  • Both GLiNER1 and GLiNER2 architectures
  • Entities carry the boxes they came from, ready to highlight or redact

CV

  • YOLO object detection, including signature and face detectors

Displaying results

  • ImageDrawBoxes / ImageCropBoxes stages, as in ScaleDP
  • showText, showNer, visualizeNer, showImage, showBoxes -- the browser equivalents of ScaleDP's notebook helpers

Installation

npm install @stabrise/scaledp

Engines are optional peer dependencies — install only what you use:

npm install pdfjs-dist                        # PDF reading
npm install onnxruntime-web ppu-paddle-ocr    # PaddleOCR
npm install onnxruntime-web @huggingface/transformers  # GLiNER NER
npm install tesseract-wasm tesseract.js       # Tesseract + script detection

Quickstart

import { Pipeline, configure } from '@stabrise/scaledp'
import { PdfToImage } from '@stabrise/scaledp/pdf'
import { PaddleTextRecognizer } from '@stabrise/scaledp/ocr'
import { GlinerNer } from '@stabrise/scaledp/ner'

configure({
  cache: 'indexeddb',
  pdf: { workerSrc: '/pdf.worker.min.mjs' },
  onProgress: (p) => console.log(`${p.file}: ${p.loaded}/${p.total}`),
})

const pipeline = new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new PaddleTextRecognizer({ preset: 'v6-small', keepFormatting: true }),
  new GlinerNer({ labels: ['person', 'organization', 'email', 'phone'] }),
])

const rows = await pipeline.transform(file)

for (const row of rows) {
  console.log(row.page, row.text.text)
  for (const entity of row.ner.entities) {
    console.log(entity.entity_group, entity.word, entity.boxes)
  }
}

Show the results

import { ImageDrawBoxes } from '@stabrise/scaledp'
import { renderInto, showText, showNer, visualizeNer } from '@stabrise/scaledp/display'

// Annotating the page is a pipeline stage, as in ScaleDP -- the result is
// just another image column.
pipeline.stages.push(
  new ImageDrawBoxes({ inputCols: ['image', 'text', 'ner'], outputCol: 'annotated' })
)

renderInto('#text', showText(row.text))            // layout-preserving text
renderInto('#entities', showNer(row.ner))          // table of entities
renderInto('#inline', visualizeNer(row.text, row.ner))  // highlighted inline

Compare with the Python original:

pipeline = PipelineModel(stages=[
    PdfDataToImage(resolution=300),
    TesseractOcr(inputCol="image", outputCol="text", keepFormatting=True),
    Ner(inputCol="text", outputCol="ner"),
])
result = pipeline.transform(df)

Skip OCR when the PDF already has text

import { PdfToDocument, hasUsableTextLayer } from '@stabrise/scaledp/pdf'

const pipeline = new Pipeline([new PdfToDocument({ resolution: 300 })])
const rows = await pipeline.transform(file)

// Only pages without a text layer need the OCR pipeline.
const needsOcr = rows.filter((r) => !hasUsableTextLayer(r.document))

Run off the main thread

// worker.ts
import { registerStages, startScaleDpWorker } from '@stabrise/scaledp/worker'
import { PdfToImage } from '@stabrise/scaledp/pdf'
import { PaddleTextRecognizer } from '@stabrise/scaledp/ocr'

registerStages({ PdfToImage, PaddleTextRecognizer })
startScaleDpWorker()
// main.ts
import { createScaleDpWorker } from '@stabrise/scaledp/worker'

const client = createScaleDpWorker({
  worker: new Worker(new URL('./worker.ts', import.meta.url), { type: 'module' }),
  onProgress: (p) => console.log(p),
})

const rows = await client.transform(
  [{ type: 'PdfToImage' }, { type: 'PaddleTextRecognizer' }],
  [{ content: bytes, path: 'invoice.pdf' }]
)

Errors never abort a pipeline

Every output schema carries an exception field. A stage that fails records the message there and the pipeline continues, so one bad page does not lose the other forty. Pass propagateError: true to a stage to make it throw instead.

const row = rows[0]
if (row.text.exception) console.warn('OCR failed:', row.text.exception)

OCR engines

Box level Models fetched WebGPU Notes
PaddleOCR (default) word ~6 MB yes 13 language presets, best all-rounder
DBNet ONNX word ~5 MB yes Detection only; mirrors ScaleDP server-side
Tesseract word ~15 MB / lang no No ONNX; good for clean Latin scans

Documentation

The docs are a site at scaledp-ts.stabrise.com, built with Fumadocs and served next to a live pipeline builder — every stage page has an Open in builder link that seeds it with a working pipeline you can run on your own file.

cd examples/vite-demo && npm install && npm run dev

License

AGPL-3.0-or-later, matching the Python library. Contact StabRise for commercial licensing.

About

ScaleDP-TS processes PDFs and images entirely in the browser: PDF rendering, text detection, OCR and named-entity recognition, composed as a pipeline. Inference runs on onnxruntime-web over WebAssembly or WebGPU.

Topics

Resources

Stars

38 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages