Data catalogue
Find datasets, series, publications and extracts across the marketplace.
- Warehouse rows
- 6.17B
- Structured DB
- 1,711.35 GB
- Live data store
- 189.73 GB
- Data tables
- 1,03,091
- Publications & data products
- 1,749
About these figures
BigQuery reports 6,167,191,751 physical rows across 89 native base tables in 9 catalogue warehouses, measured on . Structured DB measures logical bytes across the same tables. The headline excludes 22 verified staging and backup tables with 365,935,268 rows and 57.49 GB of logical storage. Separate publisher editions can contain overlapping observations.
The earlier 40-MCP inventory reported 1,115,017,091 records across warehouses, files and upstream sources, including 811,051,329 upstream survey records from MoSPI NADA.
Live data store measures 189.73 GB of live catalogue objects across 39 Cloud Storage buckets, measured on .
UNCTAD ingestion files account for 153.97 GB of additional live storage. Retained soft-deleted objects account for 109.84 GB. Raw publisher archives and their corresponding BigQuery tables are measured in their respective storage layers.
Data tables counts 1,03,091 publisher tables held across the marketplace MCPs, measured in each MCP's own store on 2026-11-10: one table, sheet, dataflow, report section or API dataset in one edition counts once, the way data.gov.in counts resources. Raw copies, backups and archive duplicates are not counted twice. They belong to 1,749 publications and data products — six Economic Survey reports are one publication — which hold 10,664 editions, 8,62,701 series and 19,717 documents. India Intelligence and Survey chapter packs are built from these MCPs and are not added. Geo-Spatial can also reach 2,36,742 data.gov.in resources live; they are not held and not counted.
The search index includes 18,933 datasets (tables, survey studies and map layers) and 493 individually indexed series. Separate live and archived access routes can have their own entries. These figures apply across the catalogue, regardless of filters.
Dataset Builder
33 results
Coverage unknown. Describe a dataset, get a multi-tab Excel workbook — retrieved from the data warehouse, official releases and filings, with the source of every cell.
build dataset
End to end in one call: spec → entities → plan → retrieve → reconcile → fill gaps → validate → workbook. Takes minutes; clients with short HTTP timeouts should use start_dataset_job + get_dataset_job. To review the spec first, call define_dataset_spec, edit, then start_dataset_job(dataset_id=...).
build workbook
Write the .xlsx: README, the spec's data tabs (derived rows as live formulas, sources in cell comments), Sources, Provenance, Conflicts, Gaps, Validation.
cancel dataset job
Cancel a local background build; keep recorded evidence for restart with start_dataset_job(dataset_id=...). Completed tasks are not retrieved again.
check dataset values
Dry-run the value gate. Return each rejection reason and normalized value without writing candidates or marking a task complete.
coverage matrix
Entity × period grid of cell status (R reported, D derived, O override, · missing) per metric.
define dataset spec
Create a dataset and its Spec: entities (rows), metrics (definition, unit, basis, domain, derive formula), periods (frequency, range) and tab layout. Pass `spec` to supply one directly (same shape as returned) instead of having it drafted from `query`. Review the result — and edit with update_dataset_spec — before retrieval: every later cost is driven by this grid.
discover sources
Connect to registered sources, list their tools, infer capabilities; optionally list candidates from the MCP registry (INSIGHT_MCP_REGISTRY_URL).
download workbook
The latest workbook: a signed 24-hour download link that opens in a browser, and, with include_base64=True, the file itself (≤ 15 MB).
estimate cost
Cells, tasks, expected tool calls and minutes for the current Spec.
fill gaps
Plan one narrower follow-up round for missing cells (must-have metrics by default) and, with run=True, execute it and re-reconcile. Cells still missing afterwards stay blank and are listed on the Gaps tab.
get dataset
Dataset state: stages run, counts, outputs, and the Spec.
get dataset evidence
Read the exact stored structured result behind a candidate's source_ref. Host-imported evidence is labelled host_supplied_result.
get dataset job
Status of a background build: status (running | done | failed), stage, progress 0-1; when done, the workbook summary, download link and validation. Long-polls up to wait_seconds (max 30) for a change; pass 0 for an immediate snapshot. A job left by a previous process is marked interrupted and can be restarted.
get retrieval tasks
Host mode: tasks with a ready agent prompt and the value schema, for your own agents and MCPs. Return their values with submit_values.
import dataset rows
Import a structured host result by column mapping, without retyping numbers. result: {columns: [...], rows: [[...]]} or {rows: [{...}]}. mapping: {entity_column|entity_constant, period_column, metrics: [{metric, column, unit}], basis, source_type, source_url, source_title, rows_path?}. Evidence is retained, every value is gated, identical retries are idempotent. Task remains open for more batches;
inspect dataset cell
Explain one cell: chosen source, all candidates/conflicts, override, formula dependencies, and persisted evidence ids. IDs and period must match the spec.
list sources
Registered sources (SQL, web search, fact stores, browsers, …) with capabilities and cost rung, plus the capability vocabulary tasks are routed by.
list values
The value ledger — every candidate an agent recorded (status: accepted | parked | all), filterable by entity id, metric id and period key.
plan retrieval
Batch the cell grid into retrieval tasks (entity × metric domain × period run) and route each to the registered sources whose capabilities fit (SQL and structured sources first). `sources` restricts this dataset to those source names.
preflight dataset
Check LLM readiness and source routing without spending retrieval calls. Reports host-mode alternatives. Use discover_sources to probe connectivity.
Inventory snapshot 15 Nov 2026. Publisher dates and individual source checks can differ.