Skip to content

Latest commit

 

History

37 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Parallel Works Workflows

The home of the Parallel Works / ACTIVATE platform workflows.

One workflow = one directory under workflows/: the variant YAMLs (in yamls/), the runtime files (controller.sh/start-template.sh + support files, in app/ or an implementation subdir — the only subtree checked out at run time), README, build tooling, and marketplace thumbnails (in thumbnails/) all live together. There are no version suffixes — every file is the latest version; git tags version the repo. Every workflow serves through the pw endpoint pattern.

How this repo was assembled, and where anything older lives, is recorded in MIGRATION.md.

Workflow index

Workflow What it serves Variants
agent-orchestrator Multi-agent orchestrator service general
hermes-agent Hermes agent with auth/TCP proxies general
jupyter Jupyter Notebook (classic) general, emed, noaa
jupyterlab JupyterLab general, emed, hsp, noaa, general_k8s, k8s
kasmvnc KasmVNC desktop (Singularity) general, general_rstudio, emed, emed_{rstudio, matlab, firefox, fsl, schrodinger, vmd}, hsp, noaa, noaa_rstudio, general_k8s, k8s
langflow-singularity Langflow (Singularity) hsp
librechat LibreChat (+ -all stacks with manager & Langflow) general, hsp, general-all, hsp-all
lite-agent Lightweight agent worker general
mlflow MLFlow on Kubernetes k8s
n8n n8n automation (Docker/Singularity) general, emed, hsp, noaa
ollama Ollama GGUF model server (native/container) general, hsp, noaa
ollama-openwebui Ollama + OpenWebUI on Kubernetes k8s
open-notebook Open Notebook (Docker) general
openvscode OpenVSCode Server general, emed, hsp, noaa, general_k8s, k8s
rag-service RAG search/index service general
rag-vllm vLLM inference server + optional RAG stack general, hsp, noaa
streamlit Streamlit apps (Singularity) general, hsp
vncserver Host desktop (KasmVNC, no container) emed, emed_{rstudio, matlab, firefox, fsl, schrodinger, vmd}
webshell Web terminal (ttyd) general, noaa

librechat-singularity-manager is a component of the librechat -all workflows (workflows/librechat/librechat-singularity-manager/).

Shared subworkflow (referenced by the workflow YAMLs via uses: github/parallelworks/workflows@canary + $yaml:):

Shared toolstools/ (tools/oras, tools/utils) are sparse-checked-out next to workflows/<name> at runtime; scripts reference them as tools/... relative to the run directory.

Tutorialstutorials/ are staged, runnable lessons that build a small demo app (demo-app/) into a full workflow: endpoint-workflows/ (the current pw endpoints pattern, stages 1–7 incl. matrix fan-out, first-start-wins, and failover) and session-workflows-hsp/ (the same journey with the older session-tunnel pattern, HSP-flavored).

Variants

general targets standard cloud/on-prem SLURM & PBS clusters. emed, hsp, and noaa are platform-deployment variants with their own schedulers, defaults, and network constraints — they can only be run from those platforms. *k8s* variants target Kubernetes clusters (docs/k8s-workflows.md).

Running a workflow

From a machine with the pw CLI authenticated:

pw workflows run "$PWD/workflows/webshell/yamls/general.yaml" -i '{"cluster":{"resource":"<cluster-name>","scheduler":false}}'

(Pass the YAML as an absolute path — the CLI parses relative paths as git hosts.)

When testing a workflow, also verify cleanup: cancel a run mid-flight and confirm nothing is left behind (processes, scheduler jobs, containers), and delete test endpoints when done — see DeveloperGuide.md.

Every workflow registers a pw endpoint (check pw endpoints list). Note that workflow YAMLs pull this repo from GitHub at runtime (parallelworks/checkout + subworkflow uses:), so local YAML edits only take full effect once pushed to the referenced branch (canary).

Developing

About

Public workflows for Parallel Works ACTIVATE

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages