forms-runner

Citizen-facing forms using a config driven Node application.

This repository is forked from the old Defra digital form builder. These projects has been adapted to run several configurations on a single instance.

DEFRA's digital form builder is a metadata-driven framework that builds on our robust, enterprise backend tech stack and the new gov.uk frontend Design System and allows form based gov.uk sites to be easily built using a graphical design tool.

The designer is no longer a plugin and is responsible for running itself on default port 3000.

• forms-runner
  • Requirements
    • Node.js
  • Local development
    • Setup
    • Development
    • Production
    • npm scripts
    • File Uploads with Local Development
    • Setting up the CDP Uploader Service
    • Configuring the Uploader URL
    • How it Works
    • Troubleshooting
  • Docker
    • Development image
    • Production image
• Environment variables
  • ⚠️ See config for default values for each environment
  • Citizen sign in
• Testing
• Outputs
• Multi-language support
  • Licence
    • About the licence

Requirements

Node.js

Please install Node.js >= v24. You will find it easier to use the Node Version Manager nvm

To use the correct version of Node.js for this application, via nvm:

$ cd forms-runner
$ nvm use

Local development

for local developers

Setup

Install application dependencies:

$ npm ci

Development

To run the application in development mode run:

$ npm run dev

Production

To mimic the application running in production mode locally run:

$ npm run start

npm scripts

All available npm scripts can be seen in package.json To view them in your command line run:

$ npm run

File Uploads with Local Development

When developing locally with file upload functionality, there are specific configuration requirements to handle CORS (Cross-Origin Resource Sharing) and CSP (Content Security Policy) restrictions.

Setting up the CDP Uploader Service

The forms-runner application needs to use the CDP uploader service which is available in the docker-compose setup from the forms-api-submissions repository:

1. Clone the forms-api-submissions repository
2. Start the required services with docker-compose:

$ cd forms-api-submissions
$ docker-compose up -d

This will start:

• The CDP uploader service on port 7337
• A Nginx reverse proxy on port 7300
• Supporting services (localstack, Redis) needed by the uploader

Configuring the Uploader URL

For file uploads to work properly in local development, you need to use the sslip.io domain that maps to the proxy:

1. Set the UPLOADER_URL in your .env file:

UPLOADER_URL=http://uploader.127.0.0.1.sslip.io:7300

2. Make sure "host.docker.internal" is enabled in your Docker Desktop settings

How it Works

• The docker-compose setup includes an Nginx reverse proxy that routes requests from uploader.127.0.0.1.sslip.io:7300 to the CDP uploader service
• When developing locally, JavaScript CORS restrictions would normally block direct requests to localhost:7337
• The sslip.io domain (a special DNS service that maps IPs to domains) allows your browser to make cross-origin requests
• The application automatically detects local development URLs with localhost:7337 and rewrites them to use the proxy

Troubleshooting

• If file uploads fail, ensure all the required docker services are running
• Verify the proxy is working by testing: http://uploader.127.0.0.1.sslip.io:7300
• Check Docker logs for the cdp-uploader and proxy containers if issues persist

Docker

Development image

Build:

$ docker build --target development --no-cache --tag forms-runner:development .

Run:

$ docker run -p 3000:3000 forms-runner:development

Production image

Build:

docker build --no-cache --tag forms-runner .

Run:

$ docker run -p 3000:3000 forms-runner

Environment variables

If there is a .env file present, these will be loaded in.

An example is shown at the bottom of this section.

⚠️ See config for default values for each environment

Please use a config file instead. This will give you more control over each environment. The defaults can be found in config. Place your config files in runner/config See https://github.com/node-config/node-config#readme for more info.

name	description	required	default	valid	notes
NODE_ENV	Node environment	no		development,test,production
PORT	Port number	no	3009
NOTIFY_TEMPLATE_ID	Notify api key	yes			Template ID required to send form payloads via GOV.UK Notify email service.
NOTIFY_API_KEY	Notify api key	yes			API KEY required to send form payloads via GOV.UK Notify email service.
LOG_LEVEL	Log level	no	debug	trace,debug,info,error
PHASE_TAG	Tag to use for phase banner	no	beta	alpha, beta, empty string
HTTP_PROXY	HTTP proxy to use, e.g. the one from CDP. Currently used for Hapi Wreck.	no
HTTPS_PROXY	HTTPS proxy to use, e.g. the one from CDP. Currently used for Hapi Wreck.	no
NO_PROXY	HTTP proxy to use, e.g. the one from CDP. Currently used for Hapi Wreck.	no
AWS_ACCESS_KEY_ID	AWS key id	yes	dummy
AWS_SECRET_ACCESS_KEY	AWS access key	yes	dummy
SNS_ENDPOINT	Endpoint for SNS messaging	yes
SNS_ADAPTER_TOPIC_ARN	The SNS topic for the submission adapter - in Amazon Resource Name (ARN) format.	yes
SNS_SAVE_TOPIC_ARN	The SNS topic for the save-and-exit - in Amazon Resource Name (ARN) format.	yes
PRIVATE_KEY_FOR_SECRETS	Base64-encoded private key (paired witht he public key from forms-manager) for decryption of secrets.	yes
USE_SIGN_IN_FEATURE	Serves citizen sign in and the per-form homepage.	no	false	true, false	See Citizen sign in.
OIDC_ISSUER	Identity provider issuer, matching the provider exactly.	when signing in
OIDC_CLIENT_ID	This service's client id at the provider.	when signing in
OIDC_REDIRECT_URI	Where the provider returns the citizen.	when signing in			The provider registers this value, so it must match this service's /auth/callback byte for byte.
OIDC_CLIENT_PRIVATE_JWK	This service's private assertion key, as a single RS256 JWK.	when signing in			Secret. The provider holds the public half.
OIDC_SUBMISSION_API_RESOURCE	Resource this service asks the provider for tokens for. It becomes the access token's audience.	no	urn:defra:forms:forms-submission-api		Must match the audience forms-submission-api accepts.
OIDC_ACCESS_TOKEN_EXPIRY_GRACE_SECONDS	An access token with this many seconds or fewer left is refreshed before use.	no	30		See Token refresh.

Citizen sign in

USE_SIGN_IN_FEATURE gates citizen sign in. It defaults to off, and with it off this service behaves as it did before the feature existed: the sign-in routes and the per-form homepage go unregistered, and the header shows no account menu.

Turn it on and OIDC_ISSUER, OIDC_REDIRECT_URI and OIDC_CLIENT_PRIVATE_JWK become required. The service reads them at boot and refuses to start while any is unset, naming the ones it wants. The other OIDC_* settings have defaults.

Sign in is an OpenID Connect authorization code flow with PKCE, against forms-identity-ui as the provider. This service proves itself with a signed assertion (private_key_jwt) rather than a shared secret, so OIDC_CLIENT_PRIVATE_JWK holds one private key and the provider is registered with the matching public half. The key's kid travels in the assertion header, which lets the provider hold both halves of a key rotation while this service signs with one.

The two halves are generated together, by node scripts/generate-client-keypair.mjs in forms-identity-ui, and must be replaced together: an assertion signed by a key the provider does not hold is rejected, and a key of a different kind from the one the code imports (RSA, RS256) fails at boot with Invalid JWK "kty" Parameter. The script prints each half as a JWKS; OIDC_CLIENT_PRIVATE_JWK takes the single key from inside the private one (printed as EXAMPLE_RP_PRIVATE_JWKS), not the surrounding { "keys": [...] }, while the provider's OIDC_RUNNER_JWKS takes the public JWKS whole.

Token refresh

Signing in stores the citizen's access, refresh and ID tokens in their session. The citizen-session auth scheme checks the access token on each request that uses it. When OIDC_ACCESS_TOKEN_EXPIRY_GRACE_SECONDS or fewer remain, it uses the refresh token to get a new access token, asking for OIDC_SUBMISSION_API_RESOURCE, and saves the result to the session. This is the only place that refreshes or saves tokens, so a route can pass request.auth.credentials.accessToken to an API without checking it. Routes that do not need the citizen, such as static assets, set auth: false and do not refresh.

The outcome of a refresh depends on the provider's answer:

• New tokens: the request carries on with the new access token.
• invalid_grant (the refresh token or its grant has expired or been revoked), or an ID token for a different citizen: the citizen is signed out. A route that requires sign in redirects to /auth/sign-in, returning to the current path after.
• Any other failure, such as the provider being unreachable: the session and tokens are kept so a later request can try again. If the current access token has not yet expired the request uses it; if it has, a route that requires sign in answers 503 Service Unavailable.

The provider does not rotate refresh tokens, so concurrent requests can refresh at the same time without invalidating each other. The session holds the refresh token, so SESSION_TIMEOUT (milliseconds) must be no shorter than the refresh token lifetime set on the provider (OIDC_TTL_REFRESH_TOKEN, in seconds). Otherwise the session, and the citizen's sign in, ends before the refresh token does.

Sign-out

Sign in sends prompt=login consent and asks for the offline_access scope. login asks for the email address and code on every sign in. consent lets the provider accept offline_access. The refresh token then works after the provider session has ended, and the provider keeps its grant at sign-out. /auth/sign-out sends the citizen to the provider's end_session_endpoint, with their ID token as the hint. The provider sends them back to /auth/signed-out:

• Completed sign-out: this service revokes the refresh token at the provider's revocation_endpoint (RFC 7009), which ends every token of the sign in. Then it removes the identity and the tokens from the session. If the revocation fails, or discovery lists no revocation_endpoint, the failure is logged, the citizen is still signed out, and the refresh token is left to expire.
• Cancelled (cancelled=true): the citizen stays signed in, with the same tokens, and goes back to the page they left.

This service reads both endpoints from the provider's discovery document once, at the first sign in, so it finds a revocation endpoint added later only after a restart.

For proxy options, see https://www.npmjs.com/package/proxy-from-env which is used by https://github.com/TooTallNate/proxy-agents/tree/main/packages/proxy-agent.

Example .env contents:

SESSION_COOKIE_PASSWORD="<cookie-password>"
SESSION_COOKIE_TTL=2419200000
REDIS_PASSWORD=my-password
REDIS_USERNAME=default
REDIS_HOST=localhost
REDIS_KEY_PREFIX=forms-runner
MANAGER_URL=http://localhost:3001
BASE_URL=http://localhost:3009
NOTIFY_TEMPLATE_ID=<notify-template-id>
NOTIFY_API_KEY=<notify-api-key>
DESIGNER_URL=http://localhost:3000
SUBMISSION_URL=http://localhost:3002
UPLOADER_BUCKET_NAME=my-bucket
UPLOADER_URL=http://uploader.127.0.0.1.sslip.io:7300
GOOGLE_TAG_MANAGER_CONTAINER_ID='GTM-XXXXXXXX'
GOOGLE_ANALYTICS_CONTAINER_ID='YYYYYYYYYY'
USE_SINGLE_INSTANCE_CACHE=true
SNS_ENDPOINT="http://localhost:4566"
SNS_ADAPTER_TOPIC_ARN="arn:aws:sns:eu-west-2:000000000000:forms_runner_submission_events"
SNS_SAVE_TOPIC_ARN="arn:aws:sns:eu-west-2:000000000000:forms_runner_events"
PRIVATE_KEY_FOR_SECRETS="<base64-encoded-private-key>"
USE_SIGN_IN_FEATURE=false
OIDC_ISSUER=http://localhost:3011
OIDC_CLIENT_ID=runner
OIDC_REDIRECT_URI=http://localhost:3009/auth/callback
OIDC_CLIENT_PRIVATE_JWK=<single-rs256-jwk>
OIDC_ACCESS_TOKEN_EXPIRY_GRACE_SECONDS=30

Testing

Tests are found inside test/cases. For test scripts, name them ${NAME}.test.js.

Outputs

At the end of a form, there are multiple output types. The schemas for the right json format can be found in the engine repo. Additional steps are required for the different output types.

• Notify
  • A GOV.UK notify is required
  • For each notification you wish to send, a template must be set up. If there are 'personalisations' they must match the configuration

Multi-language support

The runner can support multiple languages, specifically English and Welsh. See multi-language for more information.

Licence

THIS INFORMATION IS LICENSED UNDER THE CONDITIONS OF THE OPEN GOVERNMENT LICENCE found at:

http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3

The following attribution statement MUST be cited in your products and applications when using this information.

Contains public sector information licensed under the Open Government license v3

About the licence

The Open Government Licence (OGL) was developed by the Controller of Her Majesty's Stationery Office (HMSO) to enable information providers in the public sector to license the use and re-use of their information under a common open licence.

It is designed to encourage use and re-use of information freely and flexibly, with only a few conditions.