feat: generalize data download to any CoinGecko-listed token - #7
Merged
Merged
Conversation
Unify the price-data pipeline behind a single entry point. Previously download_data.py only handled Binance/Coinbase, and non-Binance tokens required copying prepare_bold_usdc_data.py each time (per the reCLAMM onboarding skill). - Add quantammsim/utils/data_processing/coingecko_data.py: the CoinGecko source (registry + resolve_cg_id, daily->1-min canonical grid with the /1440 volume spread) and rebuild_stable_peg. Hosts the single-source-of- truth COINGECKO_IDS/STABLECOINS/NATIVE_LST registries. - Wire CoinGecko into update_historic_data: source="auto" falls back to CoinGecko when the exchange waterfall finds nothing; source="coingecko" fetches directly. Extract the shared _finalize_historic_data helper so every source gets identical parquet + daily/hourly CSV + plot outputs. - download_data.py: add --source, --cg-id, --cg-days, --peg flags. The BOLD/USDC flow becomes a single command. - fetch_token_mcaps.py imports the registries from the shared module. - Remove prepare_bold_usdc_data.py; update reCLAMM SKILL.md Step 1.
mendesfabio
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
The price-data pipeline had two disjoint entry points:
download_data.py(Binance/Coinbase only) and a one-offprepare_bold_usdc_data.py(CoinGecko). The reCLAMM onboarding skill told users to copy that CoinGecko script every time a non-Binance token appeared. Every downstream consumer only cares about the canonical parquet schema, not the source — so the duplication was avoidable.What changed
quantammsim/utils/data_processing/coingecko_data.py— the CoinGecko source (registry +resolve_cg_id, daily → complete 1-minute canonical grid with the/1440volume spread) andrebuild_stable_peg. Single source of truth for theCOINGECKO_IDS/STABLECOINS/NATIVE_LSTregistries (moved out offetch_token_mcaps.py, which now imports them back).update_historic_datagainssource/cg_id/cg_days.source="auto"(default) runs the exchange waterfall and falls back to CoinGecko only if nothing is found;source="coingecko"fetches directly. The finalization block is extracted into a shared_finalize_historic_datahelper so every source produces identical parquet + daily/hourly CSV + plot outputs (CoinGecko tokens now get the daily CSV the old BOLD script never wrote).download_data.py— new--source,--cg-id,--cg-days,--peg <STABLE>flags. The BOLD/USDC flow becomes one command:prepare_bold_usdc_data.py; rewrote reCLAMM SKILL.md Step 1.Verification
== 60000msassertion./1440spread).source="coingecko"run wrote parquet (canonical columns, midnight-append), daily+hourly CSVs, and both plots with noKeyErrorfrom the new CoinGecko source color.date/close/Volume USDcolumnsmarket_featuresreads.fetch_token_mcaps.py, CLI--help, and the single-ticker guard for--cg-id/--pegall pass.The only path not exercisable offline is the live exchange → CoinGecko auto-fallback (needs a real Binance miss); that branch is a three-line guard feeding the same
get_coingecko_datathe tests cover.🤖 Generated with Claude Code