Skip to content

fix(aws): harden discovery scan data integrity and resume handling - #59

Merged
akashmaurya3160 merged 1 commit into
authsec-stagingfrom
aws-fixes-integration
Sep 24, 2026
Merged

akashmaurya3160 merged 1 commit into
authsec-stagingfrom
aws-fixes-integration

Conversation

@akashmaurya3160

Copy link
Copy Markdown

Five write-path defects in AWS discovery. Four contradict a guarantee the codebase states in its own comments, and two destroy data a rescan cannot recover.

No migrations, so rollback is a plain image revert.

The fixes

Defect Fix
B1 scanActivity discarded the count from ListIdentities, so above activityIdentityCap it read a prefix while the surface reported reached — Complete() stayed true and ReconcileGeneration deleted cloud_usage for every identity past the cap. Ordered (kind, name, id), so the same identities were starved every scan. Report partial, block reconciliation. Partial rather than denied: nothing was refused, the read stopped short.
B2 Migration 021's ARN-derived columns were written on INSERT only. Every pre-021 resource took the conflict path forever and kept is_external = false — a positive claim of locality about what may be a cross-account ARN, with idx_cloud_resource_external built to query exactly that column. Refresh resource_account, is_external, object_key. They move together because 021's CHECKs couple them to each other and to kind.
B3 iam.go says a writer "must not treat a partial role as authoritative and overwrite metadata an earlier complete read established". It did. A throttled GetRole yields Tags=nil / boundary "", both omitempty, so a blind overwrite erased the IAM tags that are the only ownership signal collected. Caller states the read fell short. Merged, not skipped — the row keeps old keys and records that this read could not confirm them.
B5 cloud_scan_run.generation is assigned at Claim and preserved across re-claims, while Scan recomputed ScanGeneration+1. They diverge when a run is re-claimed after commitScan — the crashed-worker path — stamping rows a generation ahead of the observations for the same pass. Worker passes run.Generation. Unset keeps derived behaviour.
B6 observed_at alone is not a total order, and CloudTrail rows carry e.EventTime, which AWS reports to the second. id DESC, matching the direction of the (workspace_id, subject, observed_at DESC) indexes.

Two deliberate non-changes

  • sensitivity_source / sensitivity_reason stay out of DoUpdates. They explain a verdict this scan deliberately does not refresh; updating them while sensitivity stays pinned would leave a row whose reason contradicts its value.
  • UpsertSecret has the same unconditional attrs overwrite, but upsertAccessKey never writes attrs, so the blob is always {}. Documented in place rather than fixed — a flag no caller can set is speculative API surface.

Behaviour changes to know about

  • Accounts over 500 identities now show partial and stop reconciling cloud_usage until the cap is paged. That is the safe direction — a stale row is recoverable, a deleted one is not — but it will look like a regression if unannounced.
  • B5 holds the generation steady on re-claim, which makes the previous attempt's checkpoints visible and activates the resume path that was previously unreachable. Tested end to end; nothing the first attempt wrote is lost.
  • B3 changes constraintState on a throttled re-read from last-known data rather than erased data. The merge keeps detail_incomplete accurate so it still degrades to unknown.

Verification

Eight new tests. Each fix has one that fails when the fix is removed, verified by removing it — the repo has a documented case of a test that passed with its fix removed, so this was checked rather than assumed:

  • B3 disabled → erased the owner tag: map[string]string(nil)
  • B2 disabled → columns were not refreshed: account="" is_external=false
  • B1 disabled → a truncated activity read must not license reconciliation

Three tests pin the ways a fix could be worse than the bug: a partially read role is still stamped as seen, a first sighting still lands, and an under-cap read still reconciles.

Against a live AWS account: 13 resources degraded to pre-021 defaults were repaired by a real scan — the 2 left blank are S3 ARNs, which carry no account segment. A second scan triggered through the normal UI path published with identities, permissions, 1,433 usage rows and evidence all on one generation, coverage complete.

Gates: go build, go vet, full integration suite, unit suite, migration hygiene, IGA isolation — all pass. gofmt -l flags 589 files repo-wide from CRLF line endings; these files are clean once normalised.

Scope

services/cloud_gcp_scan_identities.go is a call-site update only — it passes false and GCP behaviour is unchanged, confirmed by the GCP suite. Two unrelated ordering fixes in discovery_repository.go and governance_repository.go were deliberately dropped to keep this PR AWS-scoped; they are still open defects worth a follow-up.

Five write-path defects in AWS discovery. Four of them contradict a
guarantee the codebase states in its own comments, and two destroy data
that a rescan cannot recover.

Activity cap no longer licenses a deletion. scanActivity discarded the
row count from ListIdentities, so above activityIdentityCap it read a
prefix of the account while the surface reported `reached`. Complete()
stayed true and ReconcileGeneration deleted the cloud_usage history of
every identity past the cap -- deterministically the same identities
each scan, since the list is ordered (kind, name, id). It now reports
`partial` and blocks reconciliation. Partial, not denied: nothing was
refused, the read stopped short, and the operator should be told which.

UpsertResource refreshes migration 021's ARN-derived columns. They were
written on INSERT only, so every pre-021 resource took the conflict path
forever and kept is_external = false -- a positive claim of locality
about what may be a cross-account ARN, with a partial index built to
query exactly that column. resource_account, is_external and object_key
move together because 021's CHECKs couple them to each other and to
kind. sensitivity_source and sensitivity_reason stay out: they explain a
verdict this scan deliberately does not refresh.

UpsertIdentity stops overwriting attrs on a partial read. iam.go says a
writer "must not treat a partial role as authoritative and overwrite
metadata an earlier complete read established"; it did. A throttled
GetRole yields Tags=nil and boundary="", both omitempty, so a blind
overwrite erased the tags that are the only ownership signal collected.
The caller now says the read fell short -- detail_incomplete lives
inside the attrs JSON and is omitempty, so the repository cannot infer
it without parsing provider JSON. Merged rather than skipped, so the row
keeps the old keys AND records that this read could not confirm them;
skipping alone would leave it asserting it was fully read.

One generation authority. cloud_scan_run.generation is assigned at Claim
and preserved across re-claims, while Scan recomputed
connector.ScanGeneration+1 per attempt. They diverge when a run is
re-claimed after commitScan advanced the connector -- the crashed-worker
path -- stamping entity rows a generation ahead of the observations
written for the same pass, so evidence no longer joins to the inventory
it explains. The worker now passes run.Generation. Unset keeps the
derived behaviour for callers with no run row.

ListObservations pages over a total order. observed_at alone is not one,
and CloudTrail rows carry e.EventTime, which AWS reports to the second
-- 12 rows share a timestamp on a real account here. id DESC matches the
direction of the (workspace_id, subject, observed_at DESC) indexes.

Tests: each fix has one that fails when the fix is removed, verified by
removing it. Three more pin the ways a fix could be worse than the bug:
a partially read role is still stamped as seen, a first sighting still
lands, and an under-cap read still reconciles.

Verified against a live AWS account: 13 resources degraded to pre-021
defaults were repaired by a real scan (the 2 left blank are S3 ARNs,
which carry no account segment), and rows, permissions, usage and
evidence all published on one generation.
@akashmaurya3160 akashmaurya3160 changed the title Stop the AWS scan writing facts it did not observe fix(aws): harden discovery scan data integrity and resume handling Sep 23, 2026
@akashmaurya3160
akashmaurya3160 merged commit 6566664 into authsec-staging Sep 24, 2026
4 of 5 checks passed
adityaauthnull250401 added a commit that referenced this pull request Sep 25, 2026
Kept from #59: the ARN-derived resource columns refresh on conflict, and
observations page over (observed_at, id).

Already covered here, so this branch's version stands: the run owns the
generation (WithGeneration, T1.5); an activity read past the cap is
partial and blocks usage reconciliation (D-86); and there is no per-role
GetRole to half-fail -- roles come from the authorization details and
D-48 keeps only the keys that read does not return -- so partialRead and
its four tests are dropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant