RegStats

This repository hosts the code to collect data and generate charts and dashboards for RegStats.

Repository Structure

The project root contains several files, including an RStudio project (.Rproj), an .Rprofile configuration file, and a renv.lock file (the lockfile establishes the R environment packages used).

In addition to these files, there are several directories:

• charts/
  • contains the R code for updating RegStats charts, chart style information, and data visualization output
  • see this sub-directory's README for more details
• data/
  • contains sub-directories for each RegStats chart and dataset
  • each sub-directory contains instructions for updating each RegStats dataset (either automatically or manually), the Python code for collecting the data (when automated), and the data for each dataset
  • see the README within each sub-directory for more details
• renv/
  • contains the profiles associated with the project lockfiles; this is where package binaries will be installed on your local machine

The structure of the repository is depicted below:

Instructions for Updating Data and Charts

For updating the data or charts, follow the instructions in each subdirectory. The following list provides a summary of files and output corresponding to each data series in RegStats.

After setting up the required Python or R environment (see instructions in each subdirectory), run the Python or R code in the "File" column, and an output dataset or chart in the "Output" column will be generated.

Economically Significant Final Rules Published by Presidential Year

Update	Task	Location	File	Output
Annually: 1st week of Feb	Data	data/es_rules/	update_es_rules.py	econ_significant_rules_by_presidential_year.csv
	Chart	charts/	code/econ_significant_rules.Rmd	output/econ_significant_rules_published_by_presidential_year.pdf output/econ_significant_rules_published_by_presidential_year.png

Economically Significant Final Rules by Agency

Update	Task	Location	File	Output
Annually: 1st week of Feb	Data	data/es_rules/	by_agency/update_agency_es_rules.py	agency_econ_significant_rules_by_presidential_year.csv
	Chart	charts/	code/agency_econ_significant_rules_by_presidential_year.Rmd	output/by_agency/[agency]_econ_significant_rules_by_presidential_year.pdf output/by_agency/[agency]_econ_significant_rules_by_presidential_year.png

Monthly Significant Final Rules under the Trump 47 Administration

Update	Task	Location	File	Output
Monthly: 1st week of month (can change to subsequent admins)	Data	data/monthly_es_rules/	update_monthly_sig_rules_by_admin.py	monthly_significant_rules_by_admin.csv
	Chart	charts/	code/monthly_sig_rules_by_admin.Rmd	output/monthly_significant_rules_trump_47.pdf output/monthly_significant_rules_trump_47.png

Cumulative Economically Significant Final Rules by Administration

Update	Task	Location	File	Output
Monthly: 1st week of month	Data	data/cumulative_es_rules/	update_cumulative_es_rules.py	cumulative_econ_significant_rules_by_presidential_month.csv
	Chart	charts/	code/cumulative_econ_significant_rules_by_admin.Rmd	output/cumulative_econ_significant_rules_by_presidential_month.pdf output/cumulative_econ_significant_rules_by_presidential_month.png

Cumulative Economically Significant Final Rules by Administration in First Year

Update	Task	Location	File	Output
Monthly: 1st week of month	Data	data/cumulative_es_rules/	update_cumulative_es_rules.py	cumulative_econ_significant_rules_by_presidential_month.csv
	Chart	charts/	code/cumulative_econ_significant_rules_first_year.Rmd	output/cumulative_econ_significant_rules_by_first_year.pdf output/cumulative_econ_significant_rules_by_first_year.png

Significant Final Rules Published by Presidential Year

Update	Task	Location	File	Output
Annually: 1st week of Feb	Data	data/sig_rules/	update_sig_rules.py	significant_rules_by_presidential_year.csv
	Chart	charts/	code/significant_rules.Rmd	output/significant_rules_by_presidential_year.pdf output/significant_rules_by_presidential_year.png

Major Final Rules Published by Presidential Year

Update	Task	Location	File	Output
Annually: 1st week of Feb	Data	data/major_rules/	cradb/scraper.py cradb/process_data.py	major_rules_by_presidential_year.csv
	Chart	charts/	code/major_rules.Rmd	output/major_rules_by_presidential_year.pdf output/major_rules_by_presidential_year.png

Rules Published in the Federal Register by Presidential Year

Update	Task	Location	File	Output
Annually: 1st week of Feb	Data	data/fr_rules/	code/fr_rules_by_presidential_year.py	federal_register_rules_by_presidential_year.csv
	Chart	charts/	code/federal_register_rules.Rmd	output/federal_register_rules_by_presidential_year.pdf output/federal_register_rules_by_presidential_year.png

Rules Published in the Federal Register by Agency

Update	Task	Location	File	Output
Annually: 1st week of Feb	Data	data/fr_rules/	code/agency_fr_rules_by_presidential_year.py	agency_federal_register_rules_by_presidential_year.csv
	Chart	charts/	code/agency_federal_register_rules.Rmd	output/by_agency/[agency]_federal_register_rules_by_presidential_year.pdf output/by_agency/[agency]_federal_register_rules_by_presidential_year.png

Total Pages Published in the Code of Federal Regulations

Update	Task	Location	File	Output
Annually: during first months of calendar year	Data	data/cfr_pages/	update_cfr_pages.py	cfr_pages_by_calendar_year.csv
	Chart	charts/	code/cfr_pages.Rmd	output/cfr_pages_by_calendar_year.pdf output/cfr_pages_by_calendar_year.png

Code of Federal Regulations: Word and Page Counts by Title

Update	Task	Location	File	Output
Annually: during first months of calendar year	Data	data/cfr_pages/by_title/	scrape_cfr_by_title.py	cfr_words_pages_by_title.csv
	Chart	NA	NA	NA

Total Pages Published in the Federal Register

Update	Task	Location	File	Output
Annually: beginning of calendar year	Data	data/fr_pages/	update_fr_pages.py	federal_register_pages_by_calendar_year.csv
	Chart	charts/	code/federal_register_pages.Rmd	output/federal_register_pages_by_calendar_year.pdf output/federal_register_pages_by_calendar_year.png

Active Actions Published in the Unified Agenda

Update	Task	Location	File	Output
Biannually: spring (May/Jun) and fall (Nov/Dec)	Data	data/ua_actions/	update_ua_actions.py	active_actions_by_unified_agenda.csv
	Chart	charts/	code/unified_agenda_active_actions.Rmd	output/active_actions_by_unified_agenda.pdf output/active_actions_by_unified_agenda.png

Regulators' Budget Report: Agency Budget Outlays by Fiscal Year

Update	Task	Location	File	Output
Annually: Feb/Mar	Data	data/reg_budget/	README.md	regulatory_agency_budget_outlays_by_fy.csv
	Chart	charts/	code/reg_budget_outlays.Rmd	output/regulatory_agency_budget_outlays_by_fy.pdf output/regulatory_agency_budget_outlays_by_fy.png

Regulators' Budget Report: Full-Time Equivalent Personnel by Fiscal Year

Update	Task	Location	File	Output
Annually: Feb/Mar	Data	data/reg_budget/	README.md	regulatory_agency_personnel_by_fy.csv
	Chart	charts/	code/reg_budget_personnel.Rmd	output/regulatory_agency_personnel_by_fy.pdf output/regulatory_agency_personnel_by_fy.png

Regulators' Budget Report By Regulatory Subcategory: Agency Budget Outlays by Fiscal Year

Update	Task	Location	File	Output
Annually: Feb/Mar	Data	data/reg_budget/by_regulatory_subcategory	README.md	reg_subcategory_regulatory_agency_budget_outlays_by_fy.csv
	Chart	charts/	code/reg_subcategory_reg_budget_outlays.Rmd	output/by_regulatory_subcategory/[subcategory]_regulatory_outlays_by_fy.pdf output/by_regulatory_subcategory/[subcategory]_regulatory_outlays_by_fy.png

Regulators' Budget Report By Regulatory Subcategory: Full-Time Equivalent Personnel by Fiscal Year

Update	Task	Location	File	Output
Annually: Feb/Mar	Data	data/reg_budget/by_regulatory_subcategory	README.md	reg_subcategory_regulatory_agency_personnel_by_fy.csv
	Chart	charts/	code/reg_subcategory_reg_budget_personnel.Rmd	output/by_regulatory_subcategory/[subcategory]_regulatory_personnel_by_fy.pdf output/by_regulatory_subcategory/[subcategory]_regulatory_personnel_by_fy.png

Word Count of Public Laws by Congress

Update	Task	Location	File	Output
Biennially: 2nd week of Jan of odd years	Data	data/public_laws/	collect_public_law_data.py	public_law_word_count_by_congress.csv
	Chart	charts/	code/public_law_word_count_by_congress.Rmd	output/public_law_word_count_by_congress.pdf output/public_law_word_count_by_congress.png