Behavioral Fixes for limit and skip

[thehabes]

Summary

Resolves #301.

getPagination() never rejected anything. clampNonNegativeInt() substituted a fallback for any value it could not parse and silently capped anything above the maximum, so every bad input returned 200 with a page the client did not ask for and could not detect.

The worst case was ?limit=1e3. Number.parseInt("1e3", 10) is 1, so a client asking for a thousand records received one — and a client that stops on a short page then reports a completed walk of a single object.

All of it is fixed in one place. getPagination() in controllers/utils.js is shared by nine call sites, so /query, HEAD /query, all five /search* endpoints, and both /gog/*InManuscript endpoints change together.

What changed

Request	Before	After
?limit=1e3	1 record	400
?limit=10abc	10 records	400
?limit=250.7	250 records	400
?limit=abc, ?limit=0x10, ?limit=	100 (the default)	400
?limit=0, ?limit=-5	100 (the default)	400
?limit=100&limit=200	100	400
?limit=200&limit=100	200	400
?skip=1e3	offset 1	400
?skip=2.9	offset 2	400
?skip=abc, ?skip=-5, ?skip=	offset 0	400
?limit=1000	500, silently	500, reported in headers
?skip=150000	the page at 100000, forever	400 naming the maximum

Values are now validated as decimal integer strings before they are parsed, because Number.parseInt guesses. A parameter supplied more than once arrives from Express as an Array, which has no single correct reading, so it is refused rather than resolved by position.

The two maximums are deliberately not alike

An over-maximum limit is clamped. A page size the server can honour in part is conventional to reduce, and rejecting it would be the more breaking choice.

An over-maximum skip is rejected. Clamping it served the page at the maximum on every request past it — skip=100000, skip=100100 and skip=250000 returned byte-identical bodies — so a client advancing skip and stopping on an empty page never terminated, and accumulated the same records on every pass. There is no honest reading of it, so it is refused with a message naming the configured maximum.

Every paged response now reports what was applied

Pagination-Limit: 500
Pagination-Skip: 0
Pagination-Limit-Max: 500
Pagination-Skip-Max: 100000

The reported limit and skip are the values actually applied, which is what lets a client tell a truncated page from a genuine final one. Names are unprefixed to match the existing Annotations-Gathered and Current-Overwritten-Version headers rather than introduce a new prefix. Access-Control-Expose-Headers is already *, so browser clients can read them with no CORS change.

Environment variable names reconciled

controllers/utils.js read RERUM_MAX_QUERY_LIMIT / RERUM_MAX_QUERY_SKIP while .env sets the unprefixed MAX_QUERY_LIMIT / MAX_QUERY_SKIP. The configured caps were therefore inert and the hardcoded defaults were what actually ran. The code now reads the unprefixed names, and resolves them per call rather than capturing them at module load — which is why no test could catch the mismatch before, and why there is now one that can.

Deployment note

.env is gitignored, so this part is not in the diff and has to be done by hand.

MAX_QUERY_SKIP was bumped 10000 to 100000 locally so that honoring .env did not change the effective cap. Any deployment whose .env still reads MAX_QUERY_SKIP=10000 will see its skip cap drop from 100000 to 10000 the moment this merges, because the name mismatch that made that value inert is now fixed. Check the boxes before rolling forward.

Breaking

Requests that return 200 today return 400. Land on dev first and give known client maintainers notice.

The skip rejection is the part most likely to be hit by something real, which is the point of it. {"@type":"oa:Annotation"} still returns a document at skip=100000 on production, so the boundary is reachable by live traffic.

The published pagedQuery example in public/API.html was the non-terminating shape, so it is rewritten: it advances by the page size the server reports applying rather than by results.length, stops on a short page, and lets the boundary 400 end a walk that runs too deep.

Out of scope

• The OpenAPI contract does not document limit or skip at all today and no drift guard covers them. Left to The pagination contract is undocumented, misdocumented, and absent from the OpenAPI contract #305.
• rest.messenger appends "The body of your request was invalid… make sure it is valid JSON" to every 400, which reads oddly on a query-string error. Narrowing it means touching the shared error renderer for all 400s. The pagination messages are written to stand on their own; worth its own issue.
• HEAD /query answering an empty page with 404 (HEAD /query answers an empty page with 404 where POST /query answers 200 [] #304) is untouched. It is blocked on HEAD history - since - query #96, and the reading there is that the .head() handler should be removed from /query rather than repaired — note that /since/:_id and /history/:_id are .get() routes where HEAD is valid and should stay.