Skip to content

feat(elasticsearch): ReplicasAfterRollover and MinimumLevelOverrides options - #35

Merged
mmalkhatib merged 1 commit into
mainfrom
ahmad/log-volume-controls
Oct 25, 2026
Merged

mmalkhatib merged 1 commit into
mainfrom
ahmad/log-volume-controls

Conversation

@AhmadRAbuhussein

Copy link
Copy Markdown
Contributor

The shared Elastic cluster filled to 94% (flood stage), turning 159 indices read-only for ~27h: app logs were lost and audit writes failed. Online-selling's production app logs alone were 392 GB — mostly one EF Core Executed DbCommand line per query — and every index kept a replica.

Two opt-in SwLogger settings (unset = no behaviour change for other apps):

  • ElasticsearchReplicasAfterRollover — adds a warm phase to the managed retention policy: allocate.number_of_replicas right after the monthly rollover, with migrate disabled (no warm tier on these clusters). The index being written keeps its replicas, so node restarts don't stop logging.
  • MinimumLevelOverrides — per-source minimum levels from configuration, e.g. { "Microsoft.EntityFrameworkCore": 3 }. The Elasticsearch logger had no way to do this; the console logger already hardcodes EF Core/AspNetCore to Warning.

Tests: 5 new (policy JSON with/without the warm phase, negative value ignored, overrides filter only the configured source). Full suite 25 passed. The new policy shape was also validated against the cluster (ES 9.0.1) with a throwaway policy, since deleted.

README documents both options. Publishes 8.1.x on merge.

…options

Two opt-in SwLogger settings to keep log volume within the cluster:

- ElasticsearchReplicasAfterRollover adds a warm phase to the managed retention
  policy that sets the replica count right after a monthly rollover (migrate
  disabled: the clusters have no warm tier). The index being written keeps its
  replicas.
- MinimumLevelOverrides sets per-source minimum levels from configuration, e.g.
  Microsoft.EntityFrameworkCore at Warning, which the Elasticsearch logger had
  no way to do (the console logger already hardcodes it).

Unset, both leave behaviour unchanged.
@coderabbitai

coderabbitai Bot commented Oct 25, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Central YAML; Path: .coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 6150c7c8-ee82-42dd-b0b8-f80d14cc7313
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mmalkhatib
mmalkhatib merged commit cd8204f into main Oct 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants