Make sitemap lastmod accurate and stable - #883
Merged
Merged
Conversation
Hugo reads page dates from `git log --name-only`, and Git quotes non-ASCII paths unless core.quotepath is off; the quoted paths never matched, so every glossary term or curated resource with a non-ASCII filename (737 pages) had no lastmod. The build workflows now set core.quotepath=false. Generated collections carry their own `lastmod`: the glossary and curated resource generators keep an entry's previous date while its content is unchanged and set today's date only when it changes. Existing entries bootstrap from the file's last commit (glossary) or the sheet submission timestamp (curated resources). The data-processing workflow compares curated resources against the build-resources branch so dates do not drift between seed merges. Hugo now prefers an explicit front-matter lastmod over Git. scripts/check_sitemap_lastmod.py reports undated URLs, timestamps stamped on most of the site, and pages whose date disagrees with their source; the deploy workflow runs it as a warning-only step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAq4MsBnm8dNGe1zDqZwgn
Contributor
|
👍 All image files/references (if any) are in webp format, in line with our policy. |
Contributor
✅ Spell Check PassedNo spelling issues found when checking 8 changed file(s)! 🎉 |
Contributor
Author
|
✅ Staging Deployment Status This PR has been successfully deployed to staging as part of an aggregated deployment. Deployed at: 2026-09-02 22:22:19 UTC The staging site shows the combined state of all compatible open PRs. |
richarddushime
approved these changes
Sep 2, 2026
richarddushime
left a comment
Contributor
There was a problem hiding this comment.
TO BE MONITORED LATER
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #859
Root causes
lastmod: Hugo takes page dates fromgit log --name-only, and Git quotes paths with non-ASCII characters by default ("content/glossary/turkish/a\303\247\304\261k_bilim.md"). Hugo's parser never unquotes them, so no glossary term or curated resource with an accented or non-Latin filename got a date. Verified locally: settingcore.quotepath=falsetakes the count from 738 to 1.2026-06-11T06:17:48+08:00: that is the commit that first added the curated-resources seed copy (Update built resources #795). The deployed files come from the daily data artifact, but Hugo still looks their paths up in Git, so every resource carried the seed's commit date.Changes
deploy.yamlandstaging-aggregate.yaml:git config core.quotepath falseafter checkout (staging also getsfetch-depth: 0, matching production).config.toml:[frontmatter] lastmod = ["lastmod", ":git", "date", "publishDate"], so an explicit front-matterlastmodwins over Git.content/glossary/_create_glossaries.pyandcontent/resources/resource.pynow write alastmodkey (date only, UTC) and keep the previous value while an entry's content is unchanged. Only a real change to the entry moves the date. Shared logic inscripts/generated_lastmod.py. Bootstrap for entries that predate the key: the file's last commit date (glossary) or the sheet submissiontimestamp(curated resources).data-processing.yml: full-history checkout, and the curated generator compares against thebuild-resourcesbranch, so a changed resource does not get re-dated daily until the next seed merge.scripts/check_sitemap_lastmod.py: reports undated URLs, any timestamp stamped on more than half the site, and (with--compare-source) pages whose sitemap date disagrees with their front matter or Git history. Runs warning-only in deploy on a 200-page sample.Verification
Full local pipeline (both generators, Hugo build with quotepath off, checker on every mapped page):
lastmod/publication_types/, which has no dated members)Things to be aware of
regenerate_glossaryrun is merged.en_titleto every file. That is the last verifiable revision of each file, and it persists for terms that do not change afterwards.01/01/2020in the sheet and will show 2020-01-01 as their date until their content changes. The previous value (the June 2026 seed commit) was equally uninformative, but this one is visible on the page.lastmodin front matter and now use it instead of their Git date. Their last commits were all housekeeping (link fixes, webp conversion), so the explicit values are the better ones.🤖 Generated with Claude Code