Data catalogue
Find datasets, series, publications and extracts across the marketplace.
- Warehouse rows
- 4.67B
- Structured DB
- 1,255.05 GB
- Live data store
- 139.80 GB
- Data tables
- 1,02,754
- Publications & data products
- 1,412
About these figures
BigQuery reports 4,665,261,859 physical rows across 72 native base tables in 8 catalogue warehouses, measured on . Structured DB measures logical bytes across the same tables. The headline excludes 4 verified staging and backup tables with 359,440 rows and 0.13 GB of logical storage. Separate publisher editions can contain overlapping observations.
The earlier 40-MCP inventory reported 1,115,017,091 records across warehouses, files and upstream sources, including 811,051,329 upstream survey records from MoSPI NADA.
Live data store measures 139.80 GB of live catalogue objects across 39 Cloud Storage buckets, measured on .
UNCTAD ingestion files account for 153.97 GB of additional live storage. Retained soft-deleted objects account for 101.90 GB. Raw publisher archives and their corresponding BigQuery tables are measured in their respective storage layers.
Data tables counts 1,02,754 publisher tables held across the marketplace MCPs, measured in each MCP's own store on 2026-11-10: one table, sheet, dataflow, report section or API dataset in one edition counts once, the way data.gov.in counts resources. Raw copies, backups and archive duplicates are not counted twice. They belong to 1,412 publications and data products — six Economic Survey reports are one publication — which hold 10,664 editions, 9,269 series and 19,717 documents. India Intelligence and Survey chapter packs are built from these MCPs and are not added. Geo-Spatial can also reach 2,36,742 data.gov.in resources live; they are not held and not counted.
The search index includes 12,121 datasets (tables, survey studies and map layers) and 367 individually indexed series. Separate live and archived access routes can have their own entries. These figures apply across the catalogue, regardless of filters.
Web Harvest
9 results
Source described. Acquisition capabilities and limits from current local MCP schemas; no preloaded publisher dataset or universal website-coverage claim.
Access and runtime storage limits
CAPTCHA solving, missing credentials and unsupported dynamic protocols are explicit limits. Checkpoint resume and retained collection availability require the same accessible job-state and collection storage; Cloud Run instance-local files are not a durability guarantee.
Browse, preview and download retained files
The Files workspace lists retained originals, full derived datasets and an on-demand reader bundle separately. Preview content, inspect source URLs and checksums, and prepare authenticated original, CSV, Parquet or ZIP downloads. Prepared links expire; availability depends on this running collection and file-size budgets. Web Harvest is independent of RSI Data Extraction.
Browser interaction and dashboard protocols
Capture API traffic and execute validated browser interactions or session recipes for JavaScript, AJAX and dependent selectors. Access is conditional on configured browser services, credentials and tested protocols.
Complete Parquet tables and bounded DuckDB queries
Retain downloaded or MCP-response evidence; materialize full CSV, JSON, spreadsheet and database tables independently of previews. Catalogue and page through rows or run bounded read-only DuckDB SELECT queries.
Existing publisher MCP reader routing
Reuse operator-configured publisher MCP readers with exact source hosts, discovered schemas and series identifiers, and allowlisted read tools. Configured readers take precedence over built-in source adapters.
Extraction checks are not statistical certification
Checksums, accepted response counts and complete table conversion do not independently validate units, populations, periods or every number. Raw tables retain source quality and require source-specific semantic checks.
Portable read-only MCP bundle export
Verify retained response and Parquet hashes and row counts, then export the current collection as a read-only MCP bundle with catalogue, table, analysis, search, source and status tools.
Publisher tests cover selected source slices
Local live acceptance covers Vahan monthly comparison, NPCI product tables, a World Bank API slice and an existing RBI DBIE reader. This is not validation of every publisher, protected dashboard or historical series, and the service ships no preloaded publisher corpus.
Website acquisition and tracked extraction jobs
Plan a website or data URL, inspect access, run acquisition and track job stages, retained artifacts, failures and remaining budgets. Supports bounded extraction and crawl resume from available checkpoints.
Inventory snapshot 10 Nov 2026. Publisher dates and individual source checks can differ.