Repository navigation
Data/add 12 patent datasets - #1
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates ABDataBench documentation and benchmarking defaults while adding new patent documents to the dataset, and it improves evaluation robustness around sequence/CDRH3 matching plus VLM extraction fallbacks.
Changes:
- Refresh README/README_zh content (features, quick start, dataset links, metrics, and layout).
- Add new patent dataset markdowns (with associated images) under
dataset/. - Update benchmark judge defaults to
gzy/gemini-3.1-pro, expand rule-based sequence matching (esp. CDRH3 boundary handling), and harden VLM sequence-image extraction with a chat-completions fallback plus related config/tests.
Reviewed changes
Copilot reviewed 12 out of 896 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| README.md | Rewrites/reshapes English project overview and usage instructions. |
| README_zh.md | Mirrors README restructuring for Chinese documentation. |
| dataset/WO2018068182A1/WO2018068182A1.md | Adds a new patent document in markdown with embedded image references. |
| dataset/WO2018048939A1/WO2018048939A1.md | Adds a new patent document in markdown with embedded image references. |
| benchmark/scripts/llm_judge.py | Switches default judge model and adds rule-based sequence/CDRH3 boundary normalization logic. |
| benchmark/run_eval.py | Uses DEFAULT_MODEL from llm_judge and updates example/default model wiring. |
| benchmark/run_benchmark.sh | Updates default judge model env default. |
| benchmark/LLM_JUDGE_PROMPT.md | Updates documented default judge model. |
| agent/tools/sequence_image_tool.py | Adds configurable endpoint selection and fallback from Responses API to chat completions. |
| agent/tests/test_orchestrator_flow.py | Updates tests for _should_run_vlm_gap_fill becoming instance-based and adds new gating assertions. |
| agent/tests/test_llm_judge.py | Adds coverage for new default model behavior and new sequence matching rules. |
| agent/orchestrator.py | Gates sequence-image extractor creation and makes VLM-gap-fill decision depend on instance config state. |
| agent/config.py | Allows env-based toggling for image extraction and the sequence image tool. |
| .gitignore | Expands ignores for nested __pycache__ and *.pyc. |
| .env.example | Updates benchmark model default and clarifies judge credential fallback behavior. |
Suppressed comments (1)
agent/tools/sequence_image_tool.py:637
_auth_header_valuereturns a literal "******" when the API key is not already prefixed with "Bearer ", which will break authentication for both the Responses and chat-completions calls.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
2
to
+5
|
|
||
| [English](README.md) | [中文](README_zh.md) | ||
| [](https://github.com/GAIR-NLP/ABDataBench) | ||
| [](https://huggingface.co/datasets/GAIR/ABDataCorpus) | ||
| [](LICENSE) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.