Visuals insight

Web app for image processing, collage layout, portrait pipelines, and audio/video download + analysis. Pick a mode from the header dropdown, upload or paste a link, and export results.

Live: bluring-xi.vercel.app

Access is gated behind an access code (session cookie). Server-side features (Attention saliency, Audio/Video, Brand removal) require a valid session and the env vars below.

---

Modes

Mode	What it does
Logo blur	Draw rectangles over logos; stepped blur intensity; export PNG/JPEG
Foveal vision	Sharp centre, soft periphery — simulate central vision
Attention	MSI-Net saliency map; blur everything except predicted gaze hotspots
PowerPoint	Build a .pptx deck from blur, attention, and foveal analyses
Faces	Batch portrait pipeline: face crop, optional BG removal, ZIP export
Collages	Sort images by filename, strip whitespace, optional BG fill, ribbons or packed grids
Audio / Video	Download YouTube/Facebook (Cobalt) or Mixcloud shows, transcribe, extract slides
Brand removal	AI removes logos/text named in comma-separated filenames (2D vs 3D packaging prompts)
Text replace	Detect and replace text/numbers on flat graphics; series; Split; export naming
Išči slike	Brief (kontekst → kriterij → format → izrez) → OpenAI poizvedbe → Unsplash → ZIP

Išči slike

• Sidebar wizard: context, selection criteria, aspect ratio + long edge, crop mode, count
• Predlagaj poizvedbe via OpenAI; edit the query list before search
• Provider: Unsplash by default; Serper (Google Images) when context/criteria mentions people / osebe / ljudi
• Optional prepoznavno ime in filenames (person / EN animal / iconic role) + labels.json for captions
• Deduped results; pick up to N; crop/resize client-side; ZIP + credits.txt
• Requires OPENAI_API_KEY, UNSPLASH_ACCESS_KEY, and SERPER_API_KEY (for people searches)

Brand removal

• Upload images or ZIP (JPG, PNG, WEBP)
• Filename rule: comma-separated removal targets, e.g. packshot, ledo logo, medo maskota.jpg
• Auto classifies flat 2D graphic vs 3D product/packaging (OpenAI Vision); override in sidebar
• 3D prompts emphasize edge labels, side panels, and wrap-around packaging text
• OpenAI gpt-image-1 edit per image; download ZIP of results
• Requires OPENAI_API_KEY; optional OPENAI_BRAND_VISION_MODEL, OPENAI_BRAND_IMAGE_MODEL
• Images are resized client-side before API calls (~1536px max edge). Processing is sequential (cost/latency).

Collages

• Upload images or ZIP; natural sort (2 before 12)
• Layouts: vertical ribbon, horizontal ribbon, or collage grid
• Exact grid tilings for N images (e.g. 12 → 12×1 … 1×12); ALL builds every tiling and downloads a ZIP
• Per-row/column cell sizing (tight pack), optional gap, strip whitespace, BG removal
• PowerPoint-friendly export (1920px edge, JPEG) on by default
• Download filenames include layout, grid, pixels, and strip mode

Faces (Portraits)

• Filename ID prefix (e.g. 01_Name.jpg); batch ZIP with optional brands.json
• Smart face crop, configurable output size, background removal model picker
• Runs entirely in the browser (TensorFlow face detection + @imgly/background-removal)

Audio / Video

• Paste a YouTube or Facebook URL → choose quality (1080p, 720p, audio-only, …)
• Paste a Mixcloud show URL (mixcloud.com/user/show-slug) → audio stream (no Cobalt; resolved via Mixcloud GraphQL)
• Multi-item links show a picker when Cobalt returns several assets
• Transcription (Soniox async STT, on by default; Slovenian hint + optional speaker labels)
• Slide detection on video: export changed frames as images + PDF (both on by default). The first detected slide locks the bright projection screen; later frames are perspective-corrected to a 16∶9 rectangle (room / speaker cropped out when detection succeeds).
• Or upload a local video/audio file for analysis only

Requires a self-hosted Cobalt instance (COBALT_API_URL) for YouTube/Facebook. Mixcloud shows work without Cobalt. Public cobalt.tools is not intended for third-party apps. For production on Vercel, Cobalt must be reachable over public HTTPS (Synology reverse proxy, Cloudflare Tunnel, etc.).

---

Getting started

npm install
npm run dev

Open http://localhost:5173.

• UI only: npm run dev is enough for Logo blur, Foveal, Faces, and Collages (only /api/access is emulated in Vite).
• Audio/Video, Attention, Brand removal: plain npm run dev returns 404/503 on /api/av-* etc. Use Vercel CLI instead:

cp .env.example .env.local
# edit .env.local
npx vercel dev

Copy .env.example → .env.local and fill in values as needed.

---

Environment variables

Variable	Required for	Description
ACCESS_SECRET	Production	HMAC secret for access-session cookies
HF_TOKEN	Optional	Hugging Face token (saliency rate limits / auth)
HF_SALIENCY_SPACE_URL	Optional	Gradio Space URL (default: alexanderkroner/saliency)
COBALT_API_URL	Audio/Video downloads	Base URL of your Cobalt instance (no trailing path required)
COBALT_API_KEY	Optional	Api-Key if Cobalt auth is enabled
SONIOX_API_KEY	Audio/Video transcription	Soniox API key
SONIOX_LANGUAGE	Optional	Language hint (default sl)
SONIOX_SPEAKER_DIARIZATION	Optional	true / false (default true)
SONIOX_API_BASE_URL	Optional	Default https://api.soniox.com
OPENAI_API_KEY	Brand removal, Text replace, Išči slike	Vision / chat / query suggestions
OPENAI_BASE_URL	Optional	OpenAI-compatible API base
OPENAI_BRAND_VISION_MODEL	Optional	2D/3D classify (default gpt-4o-mini)
OPENAI_BRAND_IMAGE_MODEL	Optional	Brand edit model (default gpt-image-1)
OPENAI_IMAGE_SEARCH_MODEL	Optional	Query propose for Išči slike (default gpt-4o-mini)
UNSPLASH_ACCESS_KEY	Išči slike	Unsplash API access key (non-people)
SERPER_API_KEY	Išči slike (osebe)	Serper Google Images key
GOOGLE_CLOUD_VISION_API_KEY	Text replace	OCR geometry

See .env.example for a template.

---

Build & preview

npm run build
npm run preview

Static output goes to dist/. Vercel runs npm run build and serves dist plus serverless functions under api/.

---

Deploy

Pushes to main deploy automatically to Vercel.

1. Connect the GitHub repo in Vercel.
2. Set environment variables in the Vercel project (at minimum ACCESS_SECRET; plus Cobalt/OpenAI for Audio/Video).
3. Ensure Cobalt is on a public HTTPS URL if Audio/Video runs in production (Vercel cannot reach a LAN-only NAS IP).

Self-hosting Cobalt (Synology / Docker)

Minimal Docker Compose:

services:
  cobalt:
    image: ghcr.io/imputnet/cobalt:11
    restart: unless-stopped
    ports:
      - "9000:9000"
    environment:
      API_URL: "https://cobalt.yourdomain.example/"

Point COBALT_API_URL at that URL. Docs: run an instance, protect an instance.

---

API routes (Vercel serverless)

Route	Purpose
GET/POST /api/access	Session check / unlock with access code
POST /api/saliency	Proxy to HF Gradio saliency Space
POST /api/av-probe	Resolve media URL and quality options (Cobalt)
POST /api/av-download	Start download for chosen quality
GET /api/av-fetch	Proxy media bytes to the browser
POST /api/av-transcribe	Soniox transcription (server fetches media via downloadUrl)
POST /api/brand-analyze	Classify 2D graphic vs 3D packaging
POST /api/brand-edit	OpenAI image edit — remove listed brands
POST /api/text-detect	Vision OCR + GPT labels for Text replace
POST /api/image-search-queries	OpenAI suggests Unsplash search queries
POST /api/image-search	Unsplash or Serper search proxy (provider)
POST /api/image-search-download	Unsplash download_location ping + URL
POST /api/image-search-fetch	Proxy remote image bytes for crop/export
POST /api/image-search-labels	OpenAI recognition labels for filenames/captions

All except /api/access require a valid access cookie.

---

Stack

• Frontend: React 19, Vite, TypeScript
• Image ML (browser): TensorFlow.js face detection, @imgly/background-removal, ONNX Runtime Web
• Export: pptxgenjs, jszip, file-saver, jspdf
• Backend: Vercel serverless (Node), Hugging Face Gradio, Cobalt, Soniox, OpenAI (Vision, gpt-image-1)

---

License

Private repository — see owner for usage terms.