Skip to content

Data/add 12 patent datasets - #1

Merged
snowy2002 merged 2 commits into
mainfrom
data/add-12-patent-datasets
Aug 30, 2026
Merged

snowy2002 merged 2 commits into
mainfrom
data/add-12-patent-datasets

Conversation

@snowy2002

Copy link
Copy Markdown
Collaborator

No description provided.

Copilot AI lite review requested due to automatic review settings August 30, 2026 06:25
@snowy2002
snowy2002 merged commit c7aa6a9 into main Aug 30, 2026
1 of 3 checks passed
@snowy2002
snowy2002 deleted the data/add-12-patent-datasets branch August 30, 2026 06:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates ABDataBench documentation and benchmarking defaults while adding new patent documents to the dataset, and it improves evaluation robustness around sequence/CDRH3 matching plus VLM extraction fallbacks.

Changes:

  • Refresh README/README_zh content (features, quick start, dataset links, metrics, and layout).
  • Add new patent dataset markdowns (with associated images) under dataset/.
  • Update benchmark judge defaults to gzy/gemini-3.1-pro, expand rule-based sequence matching (esp. CDRH3 boundary handling), and harden VLM sequence-image extraction with a chat-completions fallback plus related config/tests.

Reviewed changes

Copilot reviewed 12 out of 896 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
README.md Rewrites/reshapes English project overview and usage instructions.
README_zh.md Mirrors README restructuring for Chinese documentation.
dataset/WO2018068182A1/WO2018068182A1.md Adds a new patent document in markdown with embedded image references.
dataset/WO2018048939A1/WO2018048939A1.md Adds a new patent document in markdown with embedded image references.
benchmark/scripts/llm_judge.py Switches default judge model and adds rule-based sequence/CDRH3 boundary normalization logic.
benchmark/run_eval.py Uses DEFAULT_MODEL from llm_judge and updates example/default model wiring.
benchmark/run_benchmark.sh Updates default judge model env default.
benchmark/LLM_JUDGE_PROMPT.md Updates documented default judge model.
agent/tools/sequence_image_tool.py Adds configurable endpoint selection and fallback from Responses API to chat completions.
agent/tests/test_orchestrator_flow.py Updates tests for _should_run_vlm_gap_fill becoming instance-based and adds new gating assertions.
agent/tests/test_llm_judge.py Adds coverage for new default model behavior and new sequence matching rules.
agent/orchestrator.py Gates sequence-image extractor creation and makes VLM-gap-fill decision depend on instance config state.
agent/config.py Allows env-based toggling for image extraction and the sequence image tool.
.gitignore Expands ignores for nested __pycache__ and *.pyc.
.env.example Updates benchmark model default and clarifies judge credential fallback behavior.
Suppressed comments (1)

agent/tools/sequence_image_tool.py:637

  • _auth_header_value returns a literal "******" when the API key is not already prefixed with "Bearer ", which will break authentication for both the Responses and chat-completions calls.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread README.md
Comment on lines 2 to +5

[English](README.md) | [中文](README_zh.md)
[![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github)](https://github.com/GAIR-NLP/ABDataBench)
[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97-Hugging%20Face-yellow)](https://huggingface.co/datasets/GAIR/ABDataCorpus)
[![Code License](https://img.shields.io/badge/Code-Apache_2.0-blue.svg)](LICENSE)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants