When you open a job in the editor we fetch the adaptor so we can offer autocomplete and docs. If the adaptor isn't already installed on that web pod we install it on the spot. While that install is running, a second request for the same adaptor gets handed a half-built entry, and breaks.
This is one bug with two faces, and together they account for around 656 errors in production.
What happens
Before we start installing, we add the adaptor to our in-memory list using the version string we were asked for, marked as installing, with no path yet. Anything that looks the adaptor up during that window finds that entry.
What happens next depends on the version:
- A pinned version, e.g.
@7.3.2. The entry matches, so the caller gets it back, but it has no path. The metadata code can't do anything with that and logs "AdaptorService.install failed". That's LIGHTNING-WF: 643 occurrences since February 2025, still going.
@latest. The entry's version is the literal word latest, which isn't a version number, so the comparison raises instead. That's LIGHTNING-1JE: 13 occurrences since March, still going.
Same cause, and which one you get depends only on whether the job pinned its adaptor.
The good news is it's contained to that one adaptor. The lookup checks the name before it checks the version, so a half-installed entry doesn't disturb anything else.
To see it
Two overlapping installs of an adaptor that isn't present yet:
spec = "@openfn/language-http@7.3.2"
Task.async(fn -> Lightning.AdaptorService.install(:adaptor_service, spec) end)
Process.sleep(400)
Lightning.AdaptorService.install(:adaptor_service, spec)
The second call returns {:ok, %Adaptor{status: :installing, path: nil}}, which is what produces the warning. Swap the pinned version for @latest and it raises Version.InvalidVersionError with version "latest" instead.
Worth knowing why this comes up so often: a server starts with only a handful of adaptors present, so installs aren't unusual, they're the normal path, and it starts over each time it restarts.
Why it started when it did
Every job defaults to @openfn/language-common@latest, so a great many jobs carry the word latest rather than a pinned version.
The old editor never sent that word to the adaptor service. It resolved latest into a real version number first, so the code that can't cope with it was never asked to. Restoring metadata in the new editor (#4388, 19 March) dropped that step: the channel now reads the adaptor straight off the job and passes it along as-is. The first LIGHTNING-1JE error is six days later.
The pinned-version half doesn't depend on that change, which fits it going back to February 2025.
A separate, smaller thing
LIGHTNING-22Q fired once, on 11 August, where the version asked for was the word local. That one needs a job with local actually saved against it, and we found five across two workflows on production, both since deleted. They came from internal work rather than a customer. We do check the adaptor field on write, both for shape and that the package name is one the registry knows, but the version part is deliberately permissive: the pattern explicitly allows latest and local, and in practice any word gets through. How local came to be saved on those particular jobs is still unaccounted for. It isn't from provisioning, a template or a sandbox copy, we checked all three.
Related history: when we added local adaptor support we started labelling those adaptors with the version local and never taught the metadata side about it, so metadata for local adaptors has never actually worked. Joe hit that in February in #3647.
Impact
Runs aren't affected at all. There's no retry loop. The person loses autocomplete on that job and we get a Sentry alert.
Done when
- A half-installed adaptor is never visible to a lookup, which removes both faults at once.
- A version we can't parse never raises anywhere in this path. It comes back as a normal error the channel already handles, so the person sees a clean "couldn't load metadata" and we don't get an alert. This is the safety net rather than the fix.
latest is resolved to a real version before it reaches the adaptor service, or the adaptor service handles it properly itself.- Tests cover this, which today they don't at all. There is a test file for this module but it only covers shell injection, the allowlist and package-name parsing. Wanted: two overlapping installs, and versions
latest, local, something arbitrary like next, and empty.
- LIGHTNING-WF, LIGHTNING-1JE and LIGHTNING-22Q all stop appearing.
When you open a job in the editor we fetch the adaptor so we can offer autocomplete and docs. If the adaptor isn't already installed on that web pod we install it on the spot. While that install is running, a second request for the same adaptor gets handed a half-built entry, and breaks.
This is one bug with two faces, and together they account for around 656 errors in production.
What happens
Before we start installing, we add the adaptor to our in-memory list using the version string we were asked for, marked as installing, with no path yet. Anything that looks the adaptor up during that window finds that entry.
What happens next depends on the version:
@7.3.2. The entry matches, so the caller gets it back, but it has no path. The metadata code can't do anything with that and logs "AdaptorService.install failed". That's LIGHTNING-WF: 643 occurrences since February 2025, still going.@latest. The entry's version is the literal wordlatest, which isn't a version number, so the comparison raises instead. That's LIGHTNING-1JE: 13 occurrences since March, still going.Same cause, and which one you get depends only on whether the job pinned its adaptor.
The good news is it's contained to that one adaptor. The lookup checks the name before it checks the version, so a half-installed entry doesn't disturb anything else.
To see it
Two overlapping installs of an adaptor that isn't present yet:
The second call returns
{:ok, %Adaptor{status: :installing, path: nil}}, which is what produces the warning. Swap the pinned version for@latestand it raisesVersion.InvalidVersionErrorwith version"latest"instead.Worth knowing why this comes up so often: a server starts with only a handful of adaptors present, so installs aren't unusual, they're the normal path, and it starts over each time it restarts.
Why it started when it did
Every job defaults to
@openfn/language-common@latest, so a great many jobs carry the wordlatestrather than a pinned version.The old editor never sent that word to the adaptor service. It resolved
latestinto a real version number first, so the code that can't cope with it was never asked to. Restoring metadata in the new editor (#4388, 19 March) dropped that step: the channel now reads the adaptor straight off the job and passes it along as-is. The first LIGHTNING-1JE error is six days later.The pinned-version half doesn't depend on that change, which fits it going back to February 2025.
A separate, smaller thing
LIGHTNING-22Q fired once, on 11 August, where the version asked for was the word
local. That one needs a job withlocalactually saved against it, and we found five across two workflows on production, both since deleted. They came from internal work rather than a customer. We do check the adaptor field on write, both for shape and that the package name is one the registry knows, but the version part is deliberately permissive: the pattern explicitly allowslatestandlocal, and in practice any word gets through. Howlocalcame to be saved on those particular jobs is still unaccounted for. It isn't from provisioning, a template or a sandbox copy, we checked all three.Related history: when we added local adaptor support we started labelling those adaptors with the version
localand never taught the metadata side about it, so metadata for local adaptors has never actually worked. Joe hit that in February in #3647.Impact
Runs aren't affected at all. There's no retry loop. The person loses autocomplete on that job and we get a Sentry alert.
Done when
latestis resolved to a real version before it reaches the adaptor service, or the adaptor service handles it properly itself.latest,local, something arbitrary likenext, and empty.