Academic Genealogy Graph Preprocessing Codes

A collection of scripts used for collecting, preprocessing, and enriching data for academic genealogy graph construction.

Contents

1. Web Scraping

Scripts for scraping researcher and thesis/dissertation data from academic genealogy sources (e.g. Mathematics Genealogy Project, Shodhganga).

2. Academic Genealogy Tree Preprocessing

Code for cleaning, structuring, and preparing raw scraped data into an academic genealogy tree/graph format (nodes and advisor–advisee edges).

3. Wikidata Information Extraction

Scripts to query and extract researcher-related information (affiliations, fields, biographical data, etc.) from Wikidata.

4. Python with Neo4j

Utilities for loading and querying the academic genealogy graph in Neo4j using Python, including graph construction and Cypher query helpers.

5. DBLP Parser

Code to parse the DBLP bibliography dataset for publication metadata to supplement researcher records.

6. Web Search using Google Search API

Scripts that use the Google Search API to look up and enrich researcher information (e.g. disambiguation, missing metadata) via web search.

Getting Started

Clone the repository and explore each module — see the comments/README within each subfolder for usage details specific to that component.

git clone <repo-url>
cd academic-genealogy-graph-preprocessing

Requirements

• Python 3.x
• Neo4j (for graph loading/querying scripts)
• API credentials for Google Search API (for web-search enrichment scripts)

Notes

Additional details for each component are documented within the corresponding scripts.