fix(evaluation): drop redundant duplicate_at_runtime flag so v2 runs work without manual DB patching

[Ayush8923]

Summary

Removed the DATASET_META_DUPLICATE_AT_RUNTIME / duplicate_at_runtime flag completely, it was redundant with langfuse_dataset_id IS NULL.

• S3 loader now always duplicates rows ×duplication_factor (no flag needed).
• 422 guard re-keyed onto langfuse_dataset_id, override rejected only for Langfuse-backed (v1) datasets; S3-only datasets always work.
• Upload no longer writes the flag; fixed wrong docstrings.
• Result: v2 runs work without manual DB patching; no migration needed.

Checklist

Before submitting a pull request, please ensure that you mark these task.

• Ran fastapi run --reload app/main.py or docker compose up in the repository root and test.
• If you've fixed a bug or added code that is tested and has test cases.

[coderabbitai]

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)

• ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: f7fa0c88-3cf3-404a-baf3-7cd0ce38283c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 605e34d5-a63f-4c7a-a94e-5271aef7ba9e

📥 Commits

Reviewing files that changed from the base of the PR and between 506d8b6 and bc32fc8.

📒 Files selected for processing (8)

• backend/app/crud/evaluations/dataset.py
• backend/app/services/evaluations/dataset.py
• backend/app/services/evaluations/fast.py
• backend/app/tests/api/routes/test_evaluation_dataset_v2.py
• backend/app/tests/api/routes/test_evaluation_v2.py
• backend/app/tests/services/evaluations/test_dataset_v2.py
• backend/app/tests/services/evaluations/test_load_run_dataset_items.py
• docs/wiki/modules/evaluations.md

💤 Files with no reviewable changes (2)

• backend/app/services/evaluations/dataset.py
• backend/app/crud/evaluations/dataset.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The change removes the runtime duplication metadata marker. S3-only datasets now use their stored duplication factor during fast evaluation. Langfuse-backed datasets reject duplication-factor overrides with 422. Tests and documentation reflect the updated contract.

Changes

Dataset duplication contract

Layer / File(s)	Summary
Remove runtime marker from dataset metadata backend/app/crud/evaluations/dataset.py, backend/app/services/evaluations/dataset.py	The runtime duplication constant and upload-time metadata assignment were removed.
Apply factor by dataset storage path backend/app/services/evaluations/fast.py, backend/app/tests/services/evaluations/test_load_run_dataset_items.py, backend/app/tests/api/routes/test_evaluation_v2.py	S3-only datasets always expand rows by the effective duplication factor. Langfuse-backed datasets reject duplication-factor overrides with 422.
Update duplication-factor coverage and documentation backend/app/tests/api/routes/test_evaluation_dataset_v2.py, backend/app/tests/services/evaluations/test_dataset_v2.py, docs/wiki/modules/evaluations.md	Tests and documentation now use the stored factor and the S3-only versus Langfuse-backed distinction.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DatasetUploadService
  participant DatasetMetadata
  participant FastEvaluationService
  participant ObjectStore
  DatasetUploadService->>DatasetMetadata: store duplication_factor without runtime marker
  FastEvaluationService->>DatasetMetadata: read langfuse_dataset_id and duplication_factor
  alt S3-only dataset
    FastEvaluationService->>ObjectStore: load original rows
    ObjectStore-->>FastEvaluationService: rows expanded by effective factor
  else Langfuse-backed dataset
    FastEvaluationService-->>FastEvaluationService: reject duplication_factor override with 422
  end

Loading

Merge Risk: ⚪ Minimal · up to bc32f

S3-only datasets now apply their stored duplication factor, while Langfuse-backed datasets reject duplication overrides. The updated behavior and metadata contract are covered without an outstanding merge-blocking risk.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 5 files. (1 skipped: 1…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: removing the redundant duplicate_at_runtime flag so v2 evaluation runs no longer require manual database patching.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/drop-duplication_at_runtime-flag

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

OpenAPI changes   ⚪ No API surface changes

Note

This PR does not modify the API contract.

_{main ↔ 7b9c8e7a · generated by oasdiff}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

🎉 This PR is included in version 1.7.0-main.2 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀