Data for Canada - Orthoimagery Data Packages

A collection of high-resolution orthoimagery tile packages covering Canadian cities and provinces, processed by Data for Canada from proprietary formats (ECW, MrSID, JP2) into cloud-native PMTiles. Source imagery is acquired from municipal governments and federal agencies at resolutions ranging from 7.5 cm to 30 m. Coverage includes cities such as Winnipeg, Edmonton, Vancouver, Burnaby, Red Deer, and Whitehorse, as well as provincial (Ontario) and national extents.

aerial-imagery remote-sensing pmtiles earth-observation cloud-optimized-geotiff time-series

Dataset files (61528 x .tif)

Source Cooperative product page

## README.md
# d4c-datapkg-orthoimagery

Look through our [d4c-datapkg-orthoimagery](https://github.com/dataforcanada/d4c-datapkg-orthoimagery) repo for the datasets that are currently being processed and datasets being acquired.

## Development Environment

Although we prioritize open-source tools, we currently use [MapTiler Engine Pro](https://www.maptiler.com/engine/pricing) because it outperforms available open-source alternatives for this specific workflow.

## Specifications of Tile Packages

The specifications for the tile packages are defined in this code.

```bash
#!/bin/bash
PROJECT_DIR="~/Documents/Personal/Projects/dataforcanada/d4c-datapkg-orthoimagery"
DATASET_ID="ca-mb_winnipeg-2024A00054611040_orthoimagery_2024_075mm"
DATA_DIR="${PROJECT_DIR}/data"
DATA_INPUT_DIR="${DATA_DIR}/input/${DATASET_ID}"
DATA_OUTPUT_DIR="${DATA_DIR}/output/${DATASET_ID}"

MBTILES_OUTPUT_FILE="${DATA_OUTPUT_DIR}/${DATASET_ID}.mbtiles"
PMTILES_OUTPUT_FILE="${DATA_OUTPUT_DIR}/${DATASET_ID}.pmtiles"

# Define arguments in an array
ARGS=(
  -progress
  -name "City of Winnipeg Orthoimagery for 2024 / Ortho-imagerie de la Ville de Winnipeg de 2024"
  -description "Orthoimagery 7.5cm resolution. / Ortho-imagerie à résolution de 7,5 cm."
  -attribution "Source: data.winnipeg.ca / Source: data.winnipeg.ca"
  -srs_epsg
  -mbtiles_compatible
  -wo "NUM_THREADS=ALL_CPUS"
  -wo "USE_OPENCL=TRUE"
  -sparse
  -scale 2.000000
  -work_dir ~/tmp/maptiler_engine
  -f webp32
  -webp_quality 85
  -webp_lossy
  -webp_preset photo
  -resampling cubic
  -overviews_resampling average
  -o "${MBTILES_OUTPUT_FILE}"
  $DATA_INPUT_DIR/*.ecw
)

# Run the command with the array
maptiler-engine "${ARGS[@]}"

pmtiles convert --tmpdir=~/tmp/pmtiles ${MBTILES_OUTPUT_FILE} ${PMTILES_OUTPUT_FILE}
```

## Download and Preview

Here is a table of some of the datasets created from the current process.

| Place | ISO | Year | Provider | Dataset ID & Preview | PMTiles |
| --- | --- | --- | --- | --- | --- |
| Canada | CA | 2025 | Versatiles | [ca_versatiles-2021A000011124_d4c-datapkg-orthoimagery_2025-08-10](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca_versatiles-2021A000011124_d4c-datapkg-orthoimagery_2025-08-10.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca_versatiles-2021A000011124_d4c-datapkg-orthoimagery_2025-08-10.pmtiles) |
| Canada | CA | 2020 | NRCan | [ca_nrcan-2021A000011124_d4c-datapkg-orthoimagery_2020_30m_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca_nrcan-2021A000011124_d4c-datapkg-orthoimagery_2020_30m_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca_nrcan-2021A000011124_d4c-datapkg-orthoimagery_2020_30m_v0.1.0-beta.pmtiles) |
| Edmonton | CA-AB | 2023 | Edmonton | [ca-ab_edmonton-2023A00054811061_d4c-datapkg-orthoimagery_2023_075mm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-ab_edmonton-2023A00054811061_d4c-datapkg-orthoimagery_2023_075mm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-ab_edmonton-2023A00054811061_d4c-datapkg-orthoimagery_2023_075mm_v0.1.0-beta.pmtiles) |
| Red Deer | CA-AB | 2024 | Red Deer | [ca-ab_red-deer-2024A00054808011_d4c-datapkg-orthoimagery_2024_075mm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-ab_red-deer-2024A00054808011_d4c-datapkg-orthoimagery_2024_075mm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-ab_red-deer-2024A00054808011_d4c-datapkg-orthoimagery_2024_075mm_v0.1.0-beta.pmtiles) |
| Red Deer | CA-AB | 2025 | Red Deer | [ca-ab_red-deer-2025A00054808011_d4c-datapkg-orthoimagery_2025_075mm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-ab_red-deer-2025A00054808011_d4c-datapkg-orthoimagery_2025_075mm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-ab_red-deer-2025A00054808011_d4c-datapkg-orthoimagery_2025_075mm_v0.1.0-beta.pmtiles) |
| Burnaby | CA-BC | 2020 | Burnaby | [ca-bc_burnaby-2020A00055915025_d4c-datapkg-orthoimagery_2020_075mm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-bc_burnaby-2020A00055915025_d4c-datapkg-orthoimagery_2020_075mm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-bc_burnaby-2020A00055915025_d4c-datapkg-orthoimagery_2020_075mm_v0.1.0-beta.pmtiles) |
| Vancouver | CA-BC | 2022 | Vancouver | [ca-bc_vancouver-2022A00055915022_d4c-datapkg-orthoimagery_2022_075mm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-bc_vancouver-2022A00055915022_d4c-datapkg-orthoimagery_2022_075mm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-bc_vancouver-2022A00055915022_d4c-datapkg-orthoimagery_2022_075mm_v0.1.0-beta.pmtiles) |
| Winnipeg | CA-MB | 2024 | Winnipeg | [ca-mb_winnipeg-2024A00054611040_d4c-datapkg-orthoimagery_2024_075mm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-mb_winnipeg-2024A00054611040_d4c-datapkg-orthoimagery_2024_075mm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-mb_winnipeg-2024A00054611040_d4c-datapkg-orthoimagery_2024_075mm_v0.1.0-beta.pmtiles) |
| Whitehorse | CA-YT | 2019 | Whitehorse | [ca-yt_whitehorse-2019A000556001009_d4c-datapkg-orthoimagery_2019_200mm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-yt_whitehorse-2019A000556001009_d4c-datapkg-orthoimagery_2019_200mm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-yt_whitehorse-2019A000556001009_d4c-datapkg-orthoimagery_2019_200mm_v0.1.0-beta.pmtiles) |
| Ontario | CA-ON | 2024 | Province of Ontario | [ca-on_province_of_ontario-2024A000235_drape_eastern_ontario_orthoimagery_2024_16cm_v0.1.0-beta](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-on_province_of_ontario-2024A000235_drape_eastern_ontario_orthoimagery_2024_16cm_v0.1.0-beta.pmtiles) | [Download](https://source.coop/dataforcanada/d4c-datapkg-orthoimagery/processed/ca-on_province_of_ontario-2024A000235_drape_eastern_ontario_orthoimagery_2024_16cm_v0.1.0-beta.pmtiles) |

## The Plan

The objective is to standardize the source datasets used by Data for Canada processes. For instance, Vancouver’s orthoimagery is currently distributed in proprietary formats (MrSID and ECW) that require specialized drivers. We are currently evaluating open alternatives, such as [Cloud Optimized GeoTIFFs](https://cogeo.org/), while ensuring that the conversion process preserves full visual fidelity. Future iterations will also explore the integration of multispectral data.


Data for Canada - Statistical Data Packages

Statistical data packages derived from Statistics Canada's 2021 Census of Agriculture and 2021 Population Census, covering all of Canada. Includes agricultural census variables on crop production, livestock counts, farm finances, land use, and farm operator demographics, as well as health workforce demographic tables from the population census. Data are aggregated to agricultural regions, census divisions, consolidated subdivisions, and aggregate dissemination areas, delivered in cloud-native formats (Parquet, PMTiles, FlatGeobuf).

agriculture crop-classification land-use arable-land census demographics administrative-boundaries pmtiles

processed

[placename] English name of the province or territory

[admin_code] ISO province/territory code (e.g. AB, BC, ON)

geom

[geometry] OGC CRS84 geometry for the census geographic unit (agricultural region, census division, etc.)

STRUCT(xmin FLOAT, ymin FLOAT, xmax FLOAT, ymax FLOAT)

5 files sampled for this table

processed/tables

GEO

[placename] Geographic area name for the tabular census record

DGUID

[admin_code] Statistics Canada Dissemination Geography Unique Identifier for the geographic unit

Coordinate

[latlon] Coordinate field associated with the census geographic record

Immigrant status (4):Total - Immigrant status[1]

Immigrant status (4):Non-permanent residents[4]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):Total - Major field of Study - Classification of Instructional Programs (CIP) 2021[1]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):Not applicable: less than 15 years of age[2]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):No postsecondary certificate, diploma or degree[3]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):01. Agricultural and veterinary sciences/services/operations and related fields[4]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):03. Natural resources and conservation[5]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):04. Architecture and related services[6]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):05. Area, ethnic, cultural, gender, and group studies[7]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):09. Communication, journalism and related programs[8]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):10. Communications technologies/technicians and support services[9]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):11. Computer and information sciences and support services[10]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):12. Culinary, entertainment, and personal services[11]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):13. Education[12]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):14. Engineering[13]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):15. Engineering/engineering-related technologies/technicians[14]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):16. Indigenous and foreign languages, literatures, and linguistics[15]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):19. Family and consumer sciences/human sciences[16]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):22. Legal professions and studies[17]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):23. English language and literature/letters[18]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):24. Liberal arts and sciences, general studies and humanities[19]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):25. Library science[20]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):26. Biological and biomedical sciences[21]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):27. Mathematics and statistics[22]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):28. Military science, leadership and operational art[23]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):29. Military technologies and applied sciences[24]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):30. Multidisciplinary/interdisciplinary studies[25]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):31. Parks, recreation, leisure, fitness, and kinesiology[26]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):38. Philosophy and religious studies[27]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):39. Theology and religious vocations[28]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):40. Physical sciences[29]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):41. Science technologies/technicians[30]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):42. Psychology[31]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):43. Security and protective services[32]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):44. Public administration and social service professions[33]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):45. Social sciences[34]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):46. Construction trades[35]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):47. Mechanic and repair technologies/technicians[36]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):48. Precision production[37]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):49. Transportation and materials moving[38]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):50. Visual and performing arts[39]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):51. Health professions and related programs[40]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):52. Business, management, marketing and related support services[41]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):54. History[42]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):55. French language and literature/lettersCAN[43]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):60. Health professions residency/fellowship programs[44]

Major field of Study - Classification of Instructional Programs (CIP) 2021 (45):61. Medical residency/fellowship programs[45]

English used at work (6):Total - English used at work[1]

5 files sampled for this table

Source Cooperative product page

extent.spatial.bbox: omitted; spatial reference is categorical via [processed.geom (geometry), processed.pr_iso_code (admin_code), processed.pr_en_name (placename), processed/tables.GEO (placename), processed/tables.DGUID (admin_code), processed/tables.Coordinate (latlon)]. Add bbox once the placename → bbox gazetteer covers it.


Data for Canada - Web Corpus Data Packages

A comprehensive archive of Reddit comments and submissions spanning June 2005 to December 2025, collected via the Pushshift project and subsequent community-maintained efforts. Distributed as ~3.8 TB of Zstandard-compressed NDJSON files (~80,000 files) covering the full Reddit corpus across all subreddits.

reddit social-media user-generated-content web-archive text-corpus online-communities pushshift

comments

5 files sampled for this table

submissions

5 files sampled for this table

Source Cooperative product page

## archive__reddit__subreddits25__README.md
# Reddit Comments and Submissions (2005-06 to 2025-12)

## Description
This dataset contains a comprehensive archive of Reddit comments and submissions spanning from June 2005 to December 2025.

## Dataset Details
- **Total Size:** ~3.80 TB
- **File Count:** 79,955 files
- **Format:** Zstandard compressed NDJSON (Newline Delimited JSON) files (`.zst`)
- **Source:** [Academic Torrents (Info Hash: 3d426c47c767d40f82c7ef0f47c3acacedd2bf44)](https://academictorrents.com/details/3d426c47c767d40f82c7ef0f47c3acacedd2bf44)

## Usage & Parsing
The dataset is distributed as highly compressed `.zst` files icontaining JSON objects separated by newlines (NDJSON).

Example Python scripts and tools for parsing and handling this data can be found in the following GitHub repository: **[Watchful1/PushshiftDumps](https://github.com/Watchful1/PushshiftDumps)**

## License
**No license specified.** Please note that the content consists of user-generated data scraped from Reddit. The underlying textual work may be protected by copyright. Researchers should use this data responsibly and consider Reddit's API and data usage guidelines.

## Citation
If you use this dataset in your research or academic work, you can reference it using the following BibTeX entry:

```bibtex
@misc{reddit_archive_2005_2025,
  title = {Reddit comments/submissions 2005-06 to 2025-12},
  author = {stuck_in_the_matrix and Watchful1 and RaiderBDev},
  abstract = {Reddit comments and submissions from 2005-06 to 2025-12 collected by pushshift and u/RaiderBDev. These are zstandard compressed ndjson files. Example python scripts for parsing the data can be found here [https://github.com/Watchful1/PushshiftDumps](https://github.com/Watchful1/PushshiftDumps) The more recent dumps are collected by u/RaiderBDev},
  keywords = {reddit},
  url = {[https://academictorrents.com/details/3d426c47c767d40f82c7ef0f47c3acacedd2bf44](https://academictorrents.com/details/3d426c47c767d40f82c7ef0f47c3acacedd2bf44)}
}

license: AI returned 'unknown' (README does not state a license). Verify with publisher before flipping to _state: reviewed.

providers: role [producer] for 'stuck_in_the_matrix' is an AI guess — Listed as primary author in BibTeX citation; assumed original Pushshift data collector. Verify before flipping to _state: reviewed.

providers: role [producer, processor] for 'Watchful1' is an AI guess — Listed as author; maintains PushshiftDumps tooling and assisted in collection/processing. Verify before flipping to _state: reviewed.

providers: role [producer] for 'RaiderBDev' is an AI guess — Listed as author; credited in BibTeX as collector of more recent dumps. Verify before flipping to _state: reviewed.

providers: role [host] for 'Academic Torrents' is an AI guess — Named as the source distribution channel; assumed host role. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; probe found no spatial dimension and AI did not flag any column as geographic.


GeoData-Test

No Description Provided

geo spatial
self-supervised
eatch monitoring


Kenol Fields and Buildings Dataset

This dataset contains satellite imagery collected from drones in Kenol section, Murang'a Kenya.
Dataset is intended to be labelled with field boundary and building footprint.

field boundary
building footprint
sentinel-2


Global Airport Weather GeoParquet (1940-present)

Historical and near-real-time surface weather observations from global ASOS and AWOS airport stations spanning 1940 to present, comprising approximately 1.8 billion decoded METAR reports. Each record captures temperature, dewpoint, wind speed and direction, precipitation, visibility, and atmospheric pressure in both imperial and metric units, with station point geometry embedded per record. Files are partitioned by year and stored as GeoParquet.

weather meteorology time-series geoparquet atmospheric-data climate aviation-weather

Iowa Environmental Mesonet (Iowa State University)

data

longitude

[latlon] Station longitude in decimal degrees (EPSG:4326)

latitude

[latlon] Station latitude in decimal degrees (EPSG:4326)

state

[admin_code] State or sub-national administrative code for the station location

geometry

[geometry] Station point geometry encoded as WKB (EPSG:4326)

country

[placename] Country name or code of the station

county

[placename] County or local administrative area name of the station

bbox

STRUCT(xmin DOUBLE, ymin DOUBLE, xmax DOUBLE, ymax DOUBLE)

88 files sampled for this table

Source Cooperative product page

extent.temporal: None (probe sample) widened to ['1940-01-01T00:00:00Z', '2026-01-01T00:00:00Z'] using README-stated coverage ['1940', '2026'].

extent.spatial.bbox: omitted; spatial reference is categorical via [data.longitude (latlon), data.latitude (latlon), data.geometry (geometry), data.state (admin_code), data.country (placename), data.county (placename)]. Add bbox once the placename → bbox gazetteer covers it.


DWD ICON-EU forecast, 5 day

High-resolution weather forecasts for Europe from the ICON-EU model operated by Deutscher Wetterdienst (DWD), transformed into Icechunk Zarr format by dynamical.org.


DWD ICON Weather Model Archive

Archives of Deutscher Wetterdienst (DWD, the German Meteorological Service) ICON weather model output in grib2 format. DWD maintains only a short rolling window of files on their servers, some of which we archive here.


ECMWF AIFS Single forecast

Global weather forecast dataset from ECMWF's Artificial Intelligence Forecasting System (AIFS) Single model, providing atmospheric variables at 0.25-degree global resolution. Forecasts originate from ECMWF and have been reprocessed into Zarr format by dynamical.org for efficient cloud-native access. Coverage begins April 2024 and is updated on an ongoing basis.

weather-forecasting weather numerical-weather-prediction machine-learning atmospheric-data meteorology global-coverage zarr

Dataset files (1 x .zarr)

Source Cooperative product page

## README.md
# ECMWF AIFS Single forecast - dynamical.org Zarr

See the latest dataset documentation and examples here: https://dynamical.org/catalog/ecmwf-aifs-single-forecast/


Ensemble weather forecasts from ECMWF's Integrated Forecasting System (IFS), providing 51-member probabilistic 15-day global forecasts at 0.25-degree spatial resolution with full global coverage. Forecast initialization dates are stored in cloud-optimized Zarr format, processed and openly distributed by dynamical.org.

numerical-weather-prediction ensemble-forecast weather-forecasting weather meteorology atmospheric-data global-coverage zarr

ECMWF (European Centre for Medium-Range Weather Forecasts)

Dataset files (10 x .zarr)

Source Cooperative product page

## README.md
# ECMWF IFS ENS Forecast, 15 day, 0.25 degree - dynamical.org Zarr

See the latest documentation and examples here: https://dynamical.org/catalog/ecmwf-ifs-ens-forecast-15-day-0-25-degree/


ECMWF IFS ENS Weather Model Historical Archive

Archive of ECMWF IFS ENS (Integrated Forecasting System Ensemble) historical weather forecast data in GRIB Edition 1 format at 0.25-degree global resolution. Covers approximately 2,946 forecast initialization dates from 2016-03-08 through 2024-04-01, bridging the gap between ECMWF's tape-backed MARS archive and the public S3 open-data bucket. Each date provides 51 ensemble members (1 control + 50 perturbed) with surface meteorological variables and 500 hPa geopotential at forecast steps out to 360 hours.

numerical-weather-prediction ensemble-forecast meteorology weather-forecasting weather atmospheric-data global-coverage time-series

European Centre for Medium-Range Weather Forecasts (ECMWF)

Dataset files (4211 x .grib)

Source Cooperative product page

## README.md
## ECMWF IFS ENS Forecast Archive

Archives of ensemble weather forecast data from the [European Centre for Medium-Range Weather
Forecasts](https://www.ecmwf.int/) (ECMWF).

ECMWF's public open data bucket (`ecmwf-forecasts`) only provides data from April 2024 onward.
Earlier data is stored in ECMWF's [MARS](https://www.ecmwf.int/en/forecasts/access-forecasts/access-archive-datasets)
(Meteorological Archival and Retrieval System), a tape-backed archive that requires careful request
optimization and authenticated API access.

This tool retrieves IFS ENS (Ensemble) forecast data from MARS and stages it as GRIB files in S3,
covering the period from 2016-03-08 (IFS cycle 41r2, when ENS resolution increased to 18km)
through 2024-04-01 (when S3 open data begins). The staged GRIBs can then be processed by the
[reformatters](https://github.com/dynamical-org/reformatters) into cloud-optimized Icechunk Zarr
format, available at [dynamical.org](https://dynamical.org).

The files are in GRIB Edition 1 format at 0.25 degree resolution. Each date produces four GRIB
files:

| File | Description | Approx Size |
|------|-------------|-------------|
| `pf_sfc_a.grib` | Perturbed forecast, surface, members 1-25, all steps | ~53 GB |
| `pf_sfc_b.grib` | Perturbed forecast, surface, members 26-50, all steps | ~53 GB |
| `cf_sfc.grib` | Control forecast, surface, all steps | ~2.1 GB |
| `all_pl.grib` | All members, geopotential at 500 hPa, all steps | ~9 GB |

Surface variables: `sp`, `2t`, `10u`, `10v`, `100u`, `100v`, `tp`, `strd`, `ssrd`, `msl`, `2d`,
`ptype`. Pressure level variable: `z` (geopotential) at 500 hPa. Forecast steps: 3-hourly 0-144h
and 6-hourly 150-360h (85 steps total). Ensemble members: 0 (control) + 1-50 (perturbed).

Each date totals approximately 103 GB of GRIB data. The full archive (2,946 dates) is approximately
304 TB.

The directory structure in S3 is:

`{s3_prefix}/{YYYY-MM-DD}/{request_type}.grib`

For example: `dynamical/ecmwf-ifs-grib/ecmwf-ifs-ens/2023-07-01/pf_sfc_a.grib`

The staging tool is fully resumable, tracking state in a local SQLite database. It uses ECMWF's
asynchronous MARS API, submitting requests and polling for completion. MARS requests are optimized
to maximize data per tape read: all parameters and forecast steps are requested together (same tape
per date), splitting only by ensemble member count (to stay under MARS's 75 GB retrieval limit) and
by level type (`sfc` vs `pl`).

The code is [available on GitHub](https://github.com/dynamical-org/ecmwf-ifs-backfill). The compute
to retrieve the files is provided by [dynamical.org](https://dynamical.org/).

*Data license*: ECMWF data is available under the [Creative Commons BY 4.0](https://creativecommons.org/licenses/by/4.0/)
license, provided the source is acknowledged.


NOAA GEFS analysis

An archive of the NOAA Global Ensemble Forecast System (GEFS) model's best estimate of past weather conditions, covering the globe at 0.25-degree (~20 km) resolution with 3-hourly timesteps from 2000 to present. Variables include temperature, precipitation, wind components, humidity, radiation flux, cloud cover, and precipitation type stored in Zarr format for cloud-native access. The archive is constructed by concatenating early forecast hours from three GEFS source archives: reforecasts (2000–2019), lower-resolution operational data (Jan–Sep 2020), and full-resolution operational forecasts (Sep 2020–present).

noaa meteorology weather numerical-weather-prediction ensemble-forecast global-coverage zarr atmospheric-data

NOAA National Centers for Environmental Prediction (NCEP)

Dataset files (1 x .zarr)

Source Cooperative product page

## README.md
# NOAA GEFS analysis

_View the latest documentation at https://dynamical.org/catalog/noaa-gefs-analysis/ ._

| | |
|---|---|
| Spatial domain | Global |
| Spatial resolution | 0.25 degrees (~20km) |
| Time domain | 2000-01-01 00:00:00 UTC to Present |
| Time resolution | 3.0 hours |

### Data Access
```
https://data.dynamical.org/noaa/gefs/analysis/latest.zarr?email=optional@email.com
```

*\* Email optional. Providing your email as a query param helps us understand usage and impact to keep dynamical.org supported for the long-term. For catalog updates follow [here](https://dynamical.org/updates).*

The Global Ensemble Forecast System (GEFS) is a National Oceanic and Atmospheric Administration (NOAA) National Centers for Environmental Prediction (NCEP) weather forecast model.

This analysis dataset is an archive of the model's best estimate of past weather. It is created by concatenating the first few hours of each historical forecast to provide a dataset with dimensions time, latitude, and longitude.

This dataset is designed to be used in conjunction with the [GEFS forecast 35 day](https://dynamical.org/catalog/noaa-gefs-forecast-35-day) dataset.

Storage for this dataset is generously provided by [Source Cooperative](https://source.coop/), a [Radiant Earth](https://radiant.earth/) initiative.

## Dimensions

| | min | max | units |
|---|---|---|---|
| **latitude** | -90 | 90 | degrees_north |
| **longitude** | -180 | 179.75 | degrees_east |
| **time** | 2000-01-01T00:00:00 | Present | seconds since 1970-01-01 |

## Variables

| | units | dimensions |
|---|---|---|
| **categorical_freezing_rain_surface**
*Categorical freezing rain* | 0=no; 1=yes | time × latitude × longitude |
| **categorical_ice_pellets_surface**
*Categorical ice pellets* | 0=no; 1=yes | time × latitude × longitude |
| **categorical_rain_surface**
*Categorical rain* | 0=no; 1=yes | time × latitude × longitude |
| **categorical_snow_surface**
*Categorical snow* | 0=no; 1=yes | time × latitude × longitude |
| **downward_long_wave_radiation_flux_surface**
*Surface downward long-wave radiation flux*
*Average value in the last 6 hour period (00, 06, 12, 18 UTC) or 3 hour period (03, 09, 15, 21 UTC).* | W/(m^2) | time × latitude × longitude |
| **downward_short_wave_radiation_flux_surface**
*Surface downward short-wave radiation flux*
*Average value in the last 6 hour period (00, 06, 12, 18 UTC) or 3 hour period (03, 09, 15, 21 UTC).* | W/(m^2) | time × latitude × longitude |
| **geopotential_height_cloud_ceiling**
*Geopotential height* | gpm | time × latitude × longitude |
| **maximum_temperature_2m**
*Maximum temperature* | C | time × latitude × longitude |
| **minimum_temperature_2m**
*Minimum temperature* | C | time × latitude × longitude |
| **percent_frozen_precipitation_surface**
*Percent frozen precipitation*
*Contains the value -50 when there is no precipitation.* | % | time × latitude × longitude |
| **precipitable_water_atmosphere**
*Precipitable water* | kg/(m^2) | time × latitude × longitude |
| **precipitation_surface**
*Total Precipitation*
*Average precipitation rate since the previous forecast step.* | mm/s | time × latitude × longitude |
| **pressure_reduced_to_mean_sea_level**
*Pressure reduced to MSL* | Pa | time × latitude × longitude |
| **pressure_surface**
*Surface pressure* | Pa | time × latitude × longitude |
| **relative_humidity_2m**
*2 metre relative humidity* | % | time × latitude × longitude |
| **temperature_2m**
*2 metre temperature* | C | time × latitude × longitude |
| **total_cloud_cover_atmosphere**
*Total Cloud Cover*
*Average value in the last 6 hour period (00, 06, 12, 18 UTC) or 3 hour period (03, 09, 15, 21 UTC).* | % | time × latitude × longitude |
| **wind_u_100m**
*100 metre U wind component* | m/s | time × latitude × longitude |
| **wind_u_10m**
*10 metre U wind component* | m/s | time × latitude × longitude |
| **wind_v_100m**
*100 metre V wind component* | m/s | time × latitude × longitude |
| **wind_v_10m**
*10 metre V wind component* | m/s | time × latitude × longitude |

*Don't see what you're looking for? Let us know at [feedback@dynamical.org](mailto:feedback@dynamical.org).*

## Examples

* [Open notebook in github](https://github.com/dynamical-org/notebooks/blob/main/noaa-gefs-analysis.ipynb)
* [Open notebook in colab](https://colab.research.google.com/github/dynamical-org/notebooks/blob/main/noaa-gefs-analysis.ipynb)

```python
import xarray as xr  # xarray>=2025.1.2 and zarr>=3.0.4 for zarr v3 support

ds = xr.open_zarr("https://data.dynamical.org/noaa/gefs/analysis/latest.zarr?email=optional@email.com")
ds['temperature_2m'].sel(time="2025-01-01T00", latitude=0, longitude=0).compute()
```

## Details

### Sources
To provide the longest possible historical record, this dataset in constructed from three distinct GEFS forecast archives.

- From 2000-01-01 to 2019-12-31 we use the [GEFS reforecast](https://registry.opendata.aws/noaa-gefs-reforecast/).
- From 2020-01-01 to 2020-09-23 we use [GEFS forecast archive](https://registry.opendata.aws/noaa-gefs/) data which has a lower spatial and temporal resolution.
- From 2020-09-23 to Present we use [GEFS operational forecast archives](https://registry.opendata.aws/noaa-gefs/).

### Variable availability
Data is available for all variables at all times with the following exceptions.

- Unavailable before 2020-01-01:
  `relative_humidity_2m`,
  `percent_frozen_precipitation_surface`,
  `categorical_freezing_rain_surface`,
  `categorical_ice_pellets_surface`,
  `categorical_rain_surface`,
  `categorical_snow_surface`
- Unavailable 2020-01-01T00 to 2020-09-22T21:
  `geopotential_height_cloud_ceiling`

### Construction
To create a single time dimension we concatenate the first few hours of each forecast. From 2000-01-01 to 2019-12-31 reforecasts are available once per day and this dataset uses the first 21 or 24 hours of each forecast. From 2020-01-01 to present forecasts are available every 6 hours and this dataset uses the first 3 or 6 hours of each forecast. Variables with an instantaneous `step_type` use the shortest possible lead times (e.g. 0 and 3 hours) while accumulated variables must use one additional forecast step (e.g. 3 and 6 hours) because they do not have an hour 0 forecast value.

### Interpolation
For most of the time range of the archive the source data is available at 0.25-degree resolution and a 3 hourly time step and we perform no interpolation. There are two exceptions to this. 1) From 2020-01-01 to 2020-09-23 the source data has a 1.0-degree spatial resolution and a 6 hourly time step. 2) From 2020-09-23 to present the 100m wind components have a 0.5-degree spatial resolution in the source data. To provide a consistent archive in the above two cases we first perform bilinear interpolation in space to 0.25-degree resolution followed by linear interpolation in time to a 3-hourly timestep if necessary. The original, uninterpolated data can be obtained by selecting latitudes and longitudes evenly divisible by 1 and, in case 1), time steps whose hour is divisible by 6.

### Compression
The data values in this dataset have been rounded in their binary floating point representation to improve compression. See [Klöwer et al. 2021](https://www.nature.com/articles/s43588-021-00156-2) for more information on this approach. The exact number of rounded bits can be found in our [reformatting code](https://github.com/dynamical-org/reformatters/blob/main/src/reformatters/noaa/gefs/common_gefs_template_config.py).


NOAA GEFS forecast, 35 day

Archive of NOAA Global Ensemble Forecast System (GEFS) weather forecasts produced by NOAA/NCEP, providing 31-member ensemble predictions initialized daily at 00 UTC from October 2020 to present. Forecasts extend up to 35 days (840 hours) ahead at global coverage with 0.25–0.5 degree spatial resolution, covering atmospheric variables including temperature, precipitation, wind, humidity, pressure, and radiation flux. Data are binary-rounded for improved compression and stored in cloud-optimized Zarr format by dynamical.org.

weather ensemble-forecast numerical-weather-prediction meteorology noaa global-coverage zarr precipitation

Dataset files (2 x .zarr)

Source Cooperative product page

## README.md
# NOAA GEFS forecast, 35 day - dynamical.org zarr

View the latest documentation at https://dynamical.org/catalog/noaa-gefs-forecast-35-day/.

| | |
| - | - |
| **Spatial domain**      | Global                                                                  |
| **Spatial resolution**  | 0-240 hours: 0.25 degrees (\~20km), 243-840 hours: 0.5 degrees (\~40km) |
| **Time domain**         | Forecasts initialized 2020-10-01 00:00:00 UTC to Present                |
| **Time resolution**     | Forecasts initialized every 24 hours.                                   |
| **Forecast domain**     | Forecast lead time 0-840 hours (0-35 days) ahead                        |
| **Forecast resolution** | Forecast step 0-240 hours: 3 hourly, 243-840 hours: 6 hourly            |

### Data Access

```
https://data.dynamical.org/noaa/gefs/forecast/latest.zarr?email=optional@email.com
```

*Email optional. Providing your email as a query param helps us understand usage and impact to keep dynamical.org supported for the long-term. For catalog updates follow [here](https://dynamical.org/updates).*

### Description

The Global Ensemble Forecast System (GEFS) is a National Oceanic and Atmospheric Administration (NOAA) National Centers for Environmental Prediction (NCEP) weather forecast model. GEFS creates 31 separate forecasts (ensemble members) to describe the range of forecast uncertainty.

This dataset is an archive of past and present GEFS forecasts. Forecasts are identified by an initialization time (`init_time`) denoting the start time of the model run as well as by the `ensemble_member`. Each forecast has a 3-hourly forecast step along the `lead_time` dimension. This dataset contains only the 00 hour UTC initialization times which produce the full-length, 35-day forecast.

The data values in this dataset have been rounded in their binary floating point representation to improve compression. See [Klöwer et al. 2021](https://www.nature.com/articles/s43588-021-00156-2) for more information on this approach.

Storage for this dataset is generously provided by [Source Cooperative](https://source.coop/), a [Radiant Earth](https://radiant.earth/) initiative.

## Dimensions

| Dimension            | Min                 | Max              | Units                    |
| -------------------- | ------------------- | ---------------- | ------------------------ |
| **init\_time**       | 2020-10-01T00:00:00 | Present          | seconds since 1970-01-01 |
| **ensemble\_member** | 0                   | 30               | realization              |
| **lead\_time**       | 0 days 00:00:00     | 35 days 00:00:00 | seconds                  |
| **latitude**         | -90                 | 90               | degrees\_north           |
| **longitude**        | -180                | 179.75           | degrees\_east            |

## Variables

| Variable                                         | Units               | Dimensions                                               |
|--------------------------------------------------|---------------------|---------------------------------------------------------|
| **categorical_freezing_rain_surface**            | 0=no; 1=yes         | init_time × ensemble_member × lead_time × latitude × longitude |
| **categorical_freezing_rain_surface_avg**        | 0=no; 1=yes         | init_time × lead_time × latitude × longitude           |
| **categorical_ice_pellets_surface**              | 0=no; 1=yes         | init_time × ensemble_member × lead_time × latitude × longitude |
| **categorical_ice_pellets_surface_avg**          | 0=no; 1=yes         | init_time × lead_time × latitude × longitude           |
| **categorical_rain_surface**                     | 0=no; 1=yes         | init_time × ensemble_member × lead_time × latitude × longitude |
| **categorical_rain_surface_avg**                 | 0=no; 1=yes         | init_time × lead_time × latitude × longitude           |
| **categorical_snow_surface**                     | 0=no; 1=yes         | init_time × ensemble_member × lead_time × latitude × longitude |
| **categorical_snow_surface_avg**                 | 0=no; 1=yes         | init_time × lead_time × latitude × longitude           |
| **downward_long_wave_radiation_flux_surface**    | W/(m^2)             | init_time × ensemble_member × lead_time × latitude × longitude |
| **downward_long_wave_radiation_flux_surface_avg**| W/(m^2)             | init_time × lead_time × latitude × longitude           |
| **downward_short_wave_radiation_flux_surface**   | W/(m^2)             | init_time × ensemble_member × lead_time × latitude × longitude |
| **downward_short_wave_radiation_flux_surface_avg**| W/(m^2)             | init_time × lead_time × latitude × longitude           |
| **expected_forecast_length**                     | seconds             | init_time                                              |
| **geopotential_height_cloud_ceiling**            | gpm                 | init_time × ensemble_member × lead_time × latitude × longitude |
| **ingested_forecast_length**                     | seconds             | init_time × ensemble_member                           |
| **maximum_temperature_2m**                       | C                   | init_time × ensemble_member × lead_time × latitude × longitude |
| **maximum_temperature_2m_avg**                   | C                   | init_time × lead_time × latitude × longitude           |
| **minimum_temperature_2m**                       | C                   | init_time × ensemble_member × lead_time × latitude × longitude |
| **minimum_temperature_2m_avg**                   | C                   | init_time × lead_time × latitude × longitude           |
| **percent_frozen_precipitation_surface**         | %                   | init_time × ensemble_member × lead_time × latitude × longitude |
| **precipitable_water_atmosphere**                | kg/(m^2)            | init_time × ensemble_member × lead_time × latitude × longitude |
| **precipitable_water_atmosphere_avg**            | kg/(m^2)            | init_time × lead_time × latitude × longitude           |
| **precipitation_surface**                        | kg/(m^2)            | init_time × ensemble_member × lead_time × latitude × longitude |
| **precipitation_surface_avg**                    | kg/(m^2)            | init_time × lead_time × latitude × longitude           |
| **pressure_reduced_to_mean_sea_level**           | Pa                  | init_time × ensemble_member × lead_time × latitude × longitude |
| **pressure_reduced_to_mean_sea_level_avg**       | Pa                  | init_time × lead_time × latitude × longitude           |
| **pressure_surface**                             | Pa                  | init_time × ensemble_member × lead_time × latitude × longitude |
| **pressure_surface_avg**                         | Pa                  | init_time × lead_time × latitude × longitude           |
| **relative_humidity_2m**                         | %                   | init_time × ensemble_member × lead_time × latitude × longitude |
| **relative_humidity_2m_avg**                     | %                   | init_time × lead_time × latitude × longitude           |
| **temperature_2m**                               | C                   | init_time × ensemble_member × lead_time × latitude × longitude |
| **temperature_2m_avg**                           | C                   | init_time × lead_time × latitude × longitude           |
| **total_cloud_cover_atmosphere**                 | %                   | init_time × ensemble_member × lead_time × latitude × longitude |
| **total_cloud_cover_atmosphere_avg**             | %                   | init_time × lead_time × latitude × longitude           |
| **valid_time**                                   | seconds since 1970-01-01 | init_time × lead_time                              |
| **wind_u_100m**                                  | m/s                 | init_time × ensemble_member × lead_time × latitude × longitude |
| **wind_u_10m**                                   | m/s                 | init_time × ensemble_member × lead_time × latitude × longitude |
| **wind_u_10m_avg**                               | m/s                 | init_time × lead_time × latitude × longitude           |
| **wind_v_100m**                                  | m/s                 | init_time × ensemble_member × lead_time × latitude × longitude |
| **wind_v_10m**                                   | m/s                 | init_time × ensemble_member × lead_time × latitude × longitude |
| **wind_v_10m_avg**                               | m/s                 | init_time × lead_time × latitude × longitude           |

Don't see what you're looking for? Let us know at [feedback@dynamical.org](mailto\:feedback@dynamical.org).

## Examples

[Open notebook in GitHub](https://github.com/dynamical-org/notebooks/blob/main/noaa-gefs-forecast-35-day.ipynb)

[Open notebook in Colab](https://colab.research.google.com/github/dynamical-org/notebooks/blob/main/noaa-gefs-forecast-35-day.ipynb)

```python
import xarray as xr

ds = xr.open_zarr("https://data.dynamical.org/noaa/gefs/forecast-35-day/latest.zarr?email=optional@email.com")
ds['temperature_2m'].sel(init_time="2025-01-01T00", latitude=0, longitude=0).max().compute()
```


NOAA GFS analysis, hourly

Historical weather data from the Global Forecast System (GFS) model operated by NOAA NCEP, transformed into Zarr format by dynamical.org.


NOAA GFS analysis

Global atmospheric analysis output from NOAA's Global Forecast System (GFS) model, operated by NOAA National Weather Service NCEP. Covers the full globe (180°W–179.75°E, 90°S–90°N) from May 2021 to present at sub-daily temporal resolution. Distributed in cloud-optimized Zarr format for efficient cloud-native access.

weather numerical-weather-prediction meteorology atmospheric-data noaa global-coverage zarr time-series

Dataset files (2 x .zarr)

Source Cooperative product page

## README.md
# NOAA GFS analysis - dynamical.org Zarr

See the latest dataset documentation and examples here: https://dynamical.org/catalog/noaa-gfs-analysis/


NOAA GFS forecast

Global weather forecast data from NOAA's Global Forecast System (GFS), a numerical weather prediction model producing atmospheric forecasts with full global coverage (180°W–179.75°E, 90°S–90°N). The native GFS output has been reformatted by dynamical.org into cloud-native Zarr format for efficient cloud-based access and analysis.

weather numerical-weather-prediction meteorology zarr global-coverage weather-forecasting atmospheric-modeling noaa

Dataset files (1 x .zarr)

Source Cooperative product page

## README.md
# NOAA GFS forecast - dynamical.org Zarr

See the latest dataset documentation and examples here: https://dynamical.org/catalog/noaa-gfs-forecast/


NOAA HRRR forecast, 48 hour

48-hour weather forecasts from NOAA's High Resolution Rapid Refresh (HRRR) numerical weather prediction model, covering the contiguous United States and surrounding regions. The dataset spans July 2018 to present and is derived from NOAA NWS NCEP operational model output, reformatted into cloud-optimized Zarr format by dynamical.org for efficient cloud-native access.

weather numerical-weather-prediction meteorology atmospheric-data noaa zarr weather-forecasting

Dataset files (1 x .zarr)

Source Cooperative product page

## README.md
# NOAA HRRR Forecast, 48 hour - dynamical.org Zarr

See the latest dataset documentation and examples here: https://dynamical.org/catalog/noaa-hrrr-forecast-48-hour/


NOAA MRMS CONUS analysis, hourly

Hourly precipitation analysis from NOAA's Multi-Radar Multi-Sensor (MRMS) system, operated by NWS NCEP, covering the contiguous United States (CONUS). The dataset spans from November 2014 to near-present, combining data from multiple weather radars and sensors. It has been reformatted into cloud-optimized Zarr by dynamical.org for efficient cloud access.

precipitation radar earth-observation zarr noaa meteorology weather time-series

Dataset files (2 x .zarr)

Source Cooperative product page

## README.md
# NOAA MRMS CONUS analysis, hourly - dynamical.org Zarr

See the latest dataset documentation and examples here: https://dynamical.org/catalog/noaa-mrms-conus-analysis-hourly/


East Africa Hydromet Forecasts

Pan and Pencil chunked hydromet observations datasets


East Africa Hydromet Observations


A multi-model ensemble dataset of projected future changes in water resources (total runoff, diffuse groundwater recharge, and actual evapotranspiration) across all global land areas at 0.5° resolution, derived from the ISIMIP3b simulation archive by Earth Blox. The dataset spans 4 global hydrological models × 5 CMIP6 climate models × 3 emissions scenarios (SSP1-2.6, SSP3-7.0, SSP5-8.5) across three 30-year future horizons centred on 2030, 2055, and 2085, enabling uncertainty-aware analysis of climate change impacts on hydrology.

hydrology climate zarr global-coverage ensemble-forecast numerical-weather-prediction atmospheric-modeling meteorology time-series

Dataset files (1 x .zarr)

Source Cooperative product page

## README.md
# Climate Change Impact on Water Resources (CCIWR)

A multi-model ensemble dataset of projected future changes in water resources across all global land areas, from [Earth Blox](https://earthblox.io).

<!-- Accompanying blog post: <placeholder link> -->

## General description

Climate projections of water availability are inherently uncertain: different global
hydrological models (GHMs) respond differently to the same climate forcing, and different
global climate models (GCMs) produce different forcings for the same emission scenario.
Working with a **multi-model ensemble** — rather than a single model run — is therefore
essential to characterise this uncertainty honestly and to support risk-aware adaptation
decisions. By spanning 4 GHMs × 5 GCMs × 3 SSPs, the dataset exposed here samples the
main axes of uncertainty and allows users to derive robust statistics (medians, percentiles,
model agreement) rather than relying on a potentially unrepresentative single trajectory.

This document describes the structure of the derived Zarr store built from that archive
and provides ready-to-run code snippets for common analysis patterns.

**Store location**: `s3://us-west-2.opendata.source.coop/earthblox/cciwr/data/cciwr.zarr`

---

## Store structure

The store contains **three groups**, each a separate N-dimensional array dataset:

| Group | Dimensions | Description |
|-------|-----------|-------------|
| `RC` | `(ghm, gcm, ssp, horizon, season, lat, lon)` | Relative change (%) vs. 1985–2014 baseline |
| `FUTUR` | `(ghm, gcm, ssp, horizon, season, lat, lon)` | Absolute future value (mm) averaged over the 30-yr horizon |
| `HISTO` | `(ghm, gcm, season, lat, lon)` | Historical baseline (mm) averaged over 1985–2014 |

Each group holds **three hydrological variables**:

| Variable | ISIMIP source(s) | Physical quantity |
|----------|-----------------|-------------------|
| `qr` | `qr` + `qrd` (WaterGAP only) | Diffuse groundwater recharge |
| `qtot` | `qtot` | Total runoff |
| `evap_total` | `evap-total` | Actual evapotranspiration |

**Coordinate values**

| Dimension | Values |
|-----------|--------|
| `ghm` | `cwatm`, `jules-es-vn6p3`, `miroc-integ-land`, `watergap2-2e` |
| `gcm` | `gfdl-esm4`, `ipsl-cm6a-lr`, `mpi-esm1-2-hr`, `mri-esm2-0`, `ukesm1-0-ll` |
| `ssp` | `ssp126`, `ssp370`, `ssp585` |
| `horizon` | `2030`, `2055`, `2085` (centre of 30-yr window) |
| `season` | `yearly`, `DJF`, `MAM`, `JJA`, `SON` |
| `lat` | 89.75 → −89.75 (0.5° step, **descending**) |
| `lon` | −179.75 → 179.75 (0.5° step, ascending) |

**Storage layout** (Zarr v3, sharded) — RC / FUTUR:

| Level | Shape | Role |
|-------|-------|------|
| Inner chunk | `(4 ghm, 5 gcm, 1 ssp, 1 horizon, 1 season, 20 lat, 20 lon)` | Read granularity — a point/small-AOI read fetches only the inner chunk(s) it touches |
| Shard | `(4 ghm, 5 gcm, 1 ssp, 1 horizon, 1 season, 360 lat, 720 lon)` | Storage object — one shard holds the full global map for a single ssp/horizon/season |

Requires **Zarr v3** (`zarr>=3`) to read the sharded store.

---

## Quick-start: open the store

```python
import xarray as xr
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as mcolors
import matplotlib.ticker as mticker
import cartopy.crs as ccrs
import cartopy.feature as cfeature

# ── Store location ──────────────────────────────────────────────
ZARR_PATH = 's3://us-west-2.opendata.source.coop/earthblox/cciwr/data/cciwr.zarr'

# ── Open all three groups (anonymous S3 access, lazy — no data loaded yet) ──
storage_options = {'anon': True}
ds_rc    = xr.open_zarr(ZARR_PATH, group='RC',    consolidated=True, storage_options=storage_options)
ds_futur = xr.open_zarr(ZARR_PATH, group='FUTUR', consolidated=True, storage_options=storage_options)
ds_histo = xr.open_zarr(ZARR_PATH, group='HISTO', consolidated=True, storage_options=storage_options)

ds_rc
```

---

## Case study 1 — Ensemble percentiles of groundwater recharge RC at a POI (Lyon)

Compute the **25th percentile, median, and 75th percentile** of the relative change in
groundwater recharge (`qr`) across all GHM × GCM combinations at a single point of interest,
for a user-defined SSP, horizon, and season.

The POI is snapped to the nearest 0.5° grid cell automatically.

```python
# ── User parameters ─────────────────────────────────────────────────────────
POI_LAT  = 45.75   # Lyon city centre (snapped to nearest 0.5° cell)
POI_LON  = 4.75
SSP      = 'ssp585'
HORIZON  = 2085
SEASON   = 'yearly'

# ── Select the nearest grid cell for the POI ────────────────────────────────
# lat is descending, so method='nearest' works correctly with sel()
poi_rc = ds_rc['qr'].sel(
    ssp=SSP,
    horizon=HORIZON,
    season=SEASON,
    lat=POI_LAT,
    lon=POI_LON,
)
# poi_rc shape: (ghm, gcm)  — all ensemble members at this cell

# Stack ghm × gcm into a flat ensemble dimension for quantile computation
ensemble = poi_rc.stack(member=('ghm', 'gcm'))

q10  = float(ensemble.quantile(0.10))
q25  = float(ensemble.quantile(0.25))
med  = float(ensemble.quantile(0.50))
q75  = float(ensemble.quantile(0.75))
q90  = float(ensemble.quantile(0.90))

snapped_lat, snapped_lon = float(poi_rc.lat), float(poi_rc.lon)

print(f'Scenario     : {SSP}, horizon {HORIZON}, season {SEASON}')
print(f'Variable     : qr (groundwater recharge, relative change %)')
print(f'  25th percentile : {100 * q25:+.1f} %')
print(f'  Median          : {100 * med:+.1f} %')
print(f'  75th percentile : {100 * q75:+.1f} %')
```

```python
# ── Percentile-box chart across all three horizons ───────────────────────────
# Reuses POI_LAT, POI_LON, SSP, SEASON from Case study 1 parameters.

VARIABLE = 'qr'          # variable to chart
HORIZONS       = [2030, 2055, 2085]
HORIZON_LABELS = ['2016–2045', '2041–2070', '2071–2100']

VAR_LABELS = {
    'qtot':       'Total Runoff',
    'qr':         'Groundwater Recharge',
    'evap_total': 'Actual Evapotranspiration',
}
SSP_LABELS = {
    'ssp126': 'SSP1 (RCP 2.6)',
    'ssp370': 'SSP3 (RCP 7.0)',
    'ssp585': 'SSP5 (RCP 8.5)',
}

# ── Collect percentiles for every horizon ────────────────────────────────────
horizon_stats = {}
snapped_lat_chart = snapped_lon_chart = None

for h in HORIZONS:
    # Exact match on string/categorical dims first; nearest-neighbour only for lat/lon
    poi_h = (
        ds_rc[VARIABLE]
        .sel(ssp=SSP, horizon=h, season=SEASON)
        .sel(lat=POI_LAT, lon=POI_LON, method='nearest')
    )
    ensemble_h = 100.0 * poi_h.stack(member=('ghm', 'gcm')).values

    if snapped_lat_chart is None:
        snapped_lat_chart = float(poi_h.lat)
        snapped_lon_chart = float(poi_h.lon)

    horizon_stats[h] = {
        'q10': float(np.nanpercentile(ensemble_h, 10)),
        'q25': float(np.nanpercentile(ensemble_h, 25)),
        'med': float(np.nanpercentile(ensemble_h, 50)),
        'q75': float(np.nanpercentile(ensemble_h, 75)),
        'q90': float(np.nanpercentile(ensemble_h, 90)),
        'n':   int(np.sum(~np.isnan(ensemble_h))),
    }

# ── Chart ────────────────────────────────────────────────────────────────────
fig, ax = plt.subplots(figsize=(8, 4))

x     = np.arange(len(HORIZONS))
width = 0.55

COLOR_OUTER  = '#c8c8c8'   # 10th – 90th percentile band
COLOR_IQR    = '#787878'   # 25th – 75th IQR
COLOR_MEDIAN = '#303030'   # median line

for i, h in enumerate(HORIZONS):
    s = horizon_stats[h]

    # 10th – 90th band (lightest)
    ax.bar(i, s['q90'] - s['q10'], bottom=s['q10'],
           color=COLOR_OUTER, width=width, zorder=2, linewidth=0)

    # 25th – 75th IQR (medium)
    ax.bar(i, s['q75'] - s['q25'], bottom=s['q25'],
           color=COLOR_IQR, width=width, zorder=3, linewidth=0)

    # Median line
    ax.hlines(s['med'], i - width / 2, i + width / 2,
              colors=COLOR_MEDIAN, linewidths=2.5, zorder=4)

# Reference line at 0 %
ax.axhline(0, color='black', linewidth=0.9, zorder=1)

# Axes formatting
ax.set_xticks(x)
ax.set_xticklabels(HORIZON_LABELS, fontsize=11)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda v, _: f'{v:.0f}%'))
ax.set_ylabel(
    f'Relative change in\n{VAR_LABELS.get(VARIABLE, VARIABLE)} since 1985–2014 [%]',
    fontsize=10,
)
ax.tick_params(axis='y', labelsize=10)
ax.set_xlim(-0.55, len(HORIZONS) - 0.45)
ax.set_ylim(-100, 100)

# Major + minor horizontal grid
ax.yaxis.set_minor_locator(plt.MultipleLocator(5))
ax.yaxis.grid(True, which='major', linewidth=0.5, color='lightgrey', zorder=0)
ax.yaxis.grid(True, which='minor', linewidth=0.25, color='#e8e8e8', zorder=0)
ax.set_axisbelow(True)
ax.spines[['top', 'right']].set_visible(False)

# Title
ssp_label = SSP_LABELS.get(SSP, SSP)
n_members = horizon_stats[HORIZONS[0]]['n']
ax.set_title(
    f'Lat: {snapped_lat_chart}°, Lon: {snapped_lon_chart}° – '
    f'Seasonality: {SEASON.capitalize()} – {ssp_label}',
    fontsize=11, pad=10,
)

# Footer — ensemble size + legend explanation
ax.text(
    0.5, -0.10,
    f'Based on {n_members} ensemble members',
    ha='center', va='top', transform=ax.transAxes,
    fontstyle='italic', fontsize=9,
)
ax.text(
    0.5, -0.18,
    'Outer box: 10th–90th percentile  ·  Inner box: 25th–75th percentile  ·  Line: median',
    ha='center', va='top', transform=ax.transAxes,
    fontstyle='italic', fontsize=8, color='#555555',
)

plt.tight_layout()
plt.show()
```

![Ensemble percentiles of groundwater recharge relative change at Lyon across three horizons](https://data.source.coop/earthblox/cciwr/docs/case_study_1_percentile_box.png)

---

## Case study 2 — Map of ensemble-median RC in total runoff over Europe

Compute the **ensemble median** (across all GHM × GCM combinations) of the relative change in
total runoff (`qtot`) over Europe, for a user-defined SSP, horizon, and season.

The map uses a diverging blue–white–red colormap centred on 0% to make drying vs. wetting
signals immediately visible.

```python
# ── User parameters ─────────────────────────────────────────────────────────
SSP     = 'ssp585'
HORIZON = 2085
SEASON  = 'yearly'

# Europe bounding box / Change by your area of interest.
LAT_MIN, LAT_MAX = 34.0, 72.0
LON_MIN, LON_MAX = -25.0, 45.0

# ── Spatial slice — lat is descending so min/max order is reversed ───────────
europe_rc = ds_rc['qtot'].sel(
    ssp=SSP,
    horizon=HORIZON,
    season=SEASON,
    lat=slice(LAT_MAX, LAT_MIN),   # descending axis
    lon=slice(LON_MIN, LON_MAX),
)
# europe_rc shape: (ghm, gcm, lat, lon)

# Ensemble median across all GHM × GCM pairs
median_rc = europe_rc.median(dim=['ghm', 'gcm']).compute()
# median_rc shape: (lat, lon)

# ── Diverging colormap centred on 0 ─────────────────────────────────────────
vmax = float(np.nanpercentile(np.abs(median_rc.values), 95))
norm = mcolors.TwoSlopeNorm(vmin=-vmax, vcenter=0, vmax=vmax)

# ── Map ──────────────────────────────────────────────────────────────────────
fig, ax = plt.subplots(
    figsize=(8, 7),
    subplot_kw={'projection': ccrs.PlateCarree()},
)
ax.set_extent([LON_MIN, LON_MAX, LAT_MIN, LAT_MAX], crs=ccrs.PlateCarree())

img = ax.pcolormesh(
    median_rc.lon.values,
    median_rc.lat.values,
    median_rc.values,
    norm=norm,
    cmap='RdBu',
    transform=ccrs.PlateCarree(),
)

# Overlay borders and coastlines
ax.add_feature(cfeature.BORDERS, linewidth=0.7, edgecolor='black')
ax.add_feature(cfeature.COASTLINE, linewidth=0.8)
ax.add_feature(cfeature.RIVERS, linewidth=0.4, edgecolor='steelblue', alpha=0.6)
ax.gridlines(draw_labels=False, linewidth=0.3, color='grey', alpha=0.5)

cbar = plt.colorbar(img, ax=ax, orientation='vertical', pad=0.02, fraction=0.03)
cbar.set_label('Relative change in total runoff', fontsize=10)
cbar.formatter = mticker.PercentFormatter(xmax=1.0, decimals=0)
cbar.update_ticks()

ax.set_title(
    f'Ensemble-median RC — qtot | {SSP}, horizon {HORIZON}, {SEASON}\n'
    f'(n = {len(ds_rc.ghm) * len(ds_rc.gcm)} ensemble members)',
    fontsize=11,
)

plt.tight_layout()
plt.show()
```

![Ensemble-median relative change in total runoff over Europe](https://data.source.coop/earthblox/cciwr/docs/case_study_2_europe_median_runoff.png)

---

## Case study 3 — Model consensus on a drying threshold (groundwater recharge)

The ensemble median tells you the central signal, but not how *confident* the models are.
A more decision-relevant question is: **for a given level of stress, how many of the
ensemble members agree that it will be reached?**

Here we map **model consensus** over Europe — the percentage of the GHM × GCM members
that project a relative change in groundwater recharge (`qr`) **below a chosen drying
threshold**. The threshold is set with the `THRESHOLD` parameter at the top of the next cell
(default: **−20%**). A value of 100% means *every* model agrees the cell dries by at least
that much; 0% means *no* model does. Note that the number of available members can vary from
pixel to pixel, so the consensus is normalised by the valid members at each cell rather than
a fixed count. This turns the raw ensemble into a transparent, risk-oriented agreement map,
using a sequential colormap where darker = stronger consensus on drying.

```python
# ── User parameters ─────────────────────────────────────────────────────────
SSP       = 'ssp585'
HORIZON   = 2085
SEASON    = 'yearly'    # could also do seasonal slices (e.g. 'DJF', 'MAM', 'JJA', 'SON') for a more seasonal perspective
VARIABLE  = 'qr'        # groundwater recharge
THRESHOLD = -0.20       # drying stress: count members with RC below -20%

# Europe bounding box (same extent as Case study 2).
LAT_MIN, LAT_MAX = 34.0, 72.0
LON_MIN, LON_MAX = -25.0, 45.0

# ── Spatial slice — lat is descending so min/max order is reversed ───────────
europe_rc = ds_rc[VARIABLE].sel(
    ssp=SSP,
    horizon=HORIZON,
    season=SEASON,
    lat=slice(LAT_MAX, LAT_MIN),   # descending axis
    lon=slice(LON_MIN, LON_MAX),
)
# europe_rc shape: (ghm, gcm, lat, lon)

# ── Model consensus: fraction of members with RC below the drying threshold ──
# Count members satisfying the stress, ignoring NaNs, then divide by valid count.
# The number of valid members varies from pixel to pixel (it is NOT always 20),
# so we normalise by the per-pixel valid count rather than a fixed denominator.
below     = (europe_rc < THRESHOLD)
n_below   = below.sum(dim=['ghm', 'gcm'])
n_valid   = europe_rc.notnull().sum(dim=['ghm', 'gcm'])
consensus = (n_below / n_valid).where(n_valid > 0).compute()   # 0–1, NaN where no data
# consensus shape: (lat, lon)

# ── Sequential colormap (0–100% agreement) ──────────────────────────────────
norm = mcolors.Normalize(vmin=0.0, vmax=1.0)

# ── Map ──────────────────────────────────────────────────────────────────────
fig, ax = plt.subplots(
    figsize=(8, 7),
    subplot_kw={'projection': ccrs.PlateCarree()},
)
ax.set_extent([LON_MIN, LON_MAX, LAT_MIN, LAT_MAX], crs=ccrs.PlateCarree())

img = ax.pcolormesh(
    consensus.lon.values,
    consensus.lat.values,
    consensus.values,
    norm=norm,
    cmap='YlOrRd',
    transform=ccrs.PlateCarree(),
)

# Overlay borders and coastlines
ax.add_feature(cfeature.BORDERS, linewidth=0.7, edgecolor='black')
ax.add_feature(cfeature.COASTLINE, linewidth=0.8)
ax.add_feature(cfeature.RIVERS, linewidth=0.4, edgecolor='steelblue', alpha=0.6)
ax.gridlines(draw_labels=False, linewidth=0.3, color='grey', alpha=0.5)

cbar = plt.colorbar(img, ax=ax, orientation='vertical', pad=0.02, fraction=0.03)
cbar.set_label(f'Model consensus on RC < {THRESHOLD:+.0%} (share of members)', fontsize=10)
cbar.formatter = mticker.PercentFormatter(xmax=1.0, decimals=0)
cbar.update_ticks()

ax.set_title(
    f'Model consensus on drying — {VAR_LABELS.get(VARIABLE, VARIABLE)} '
    f'RC < {THRESHOLD:+.0%}\n'
    f'{SSP}, horizon {HORIZON}, {SEASON}',
    fontsize=11,
)

plt.tight_layout()
plt.show()
```

![Model consensus on drying of groundwater recharge over Europe](https://data.source.coop/earthblox/cciwr/docs/case_study_3_europe_consensus_drying.png)

---

## Resources & references

### Underlying source data — ISIMIP3b
This composite dataset is derived entirely from the **ISIMIP3b** (Inter-Sectoral Impact Model
Intercomparison Project, phase 3b) global water-sector simulation archive. Monthly time series
of total runoff (`qtot`), diffuse groundwater recharge (`qr` / `qrd`), and actual
evapotranspiration (`evap-total`) were downloaded from the ISIMIP data repository
([https://data.isimip.org/](https://data.isimip.org/)) under the `2015soc` experiment
([protocol](https://protocol.isimip.org/#/ISIMIP3b/water_global)). The historical reference
period uses the `historical_histsoc_default` specifier; future projections use the
`ssp126_2015soc-from-histsoc_default`, `ssp370_2015soc-from-histsoc_default`, and
`ssp585_2015soc-from-histsoc_default` specifiers for SSP1-2.6, SSP3-7.0, and SSP5-8.5
respectively. The full simulation data are cited as:

> Gosling, S. N., Müller Schmied, H., Bradley, A., Burek, P., Gedney, N., Grillakis, M.,
> Guillaumot, L., Hanasaki, N., Ito, A., Kou-Giesbrecht, S., Koutroulis, A., Nishina, K.,
> Otta, K., Sahu, R.-K., Satoh, Y., and Schewe, J. (2024).
> *ISIMIP3b Simulation Data from the Global Water Sector (v1.3).*
> ISIMIP Repository. DOI: [10.48364/ISIMIP.230418.3](https://doi.org/10.48364/ISIMIP.230418.3)

### Paper

Döll, P., Attard, G., Kneier, F., and Müller, L. (2026).
**Supporting climate change adaptation worldwide: A web application for exploring uncertain future changes in water resources.** *EGUsphere* [preprint], under review for *Geoscience Communication*.
DOI: [10.5194/egusphere-2026-1829](https://doi.org/10.5194/egusphere-2026-1829)

### Interactive explorer

The **CCIWR Explorer** is a free Earth Engine web application that lets you visualise
ensemble projections of total water resources, groundwater recharge, and evapotranspiration
across four seasons and three SSP scenarios — including percentile boxes for individual 0.5° grid cells.[https://ee-gwp.projects.earthengine.app/view/cciwr-explorer](https://ee-gwp.projects.earthengine.app/view/cciwr-explorer)

providers: role [producer] for 'ISIMIP Repository' is an AI guess — ISIMIP3b produced the underlying global hydrological model simulation data from which this dataset is derived; treated as producer of the source data.. Verify before flipping to _state: reviewed.

extent.temporal: None (probe sample) widened to ['1985-01-01T00:00:00Z', '2100-01-01T00:00:00Z'] using README-stated coverage ['1985', '2100'].


Amazon Mining Watch

Amazon Mining Watch is a dataset of automated machine-learning detections of artisanal gold mine scars derived from Sentinel-2 satellite imagery, covering all nine Amazonian countries. Detections are stored as cumulative GeoJSON polygons at yearly granularity from 2018–2024 and quarterly from Q2 2025. The dataset enables tracking of legal and illegal gold mining expansion across the Amazon rainforest.

gold-mining amazon illegal-mining remote-sensing machine-learning environmental-monitoring deforestation conservation

Dataset files (10 x .geojson)

Source Cooperative product page

## README.md

![EarthGenome](https://data.source.coop/earthgenome/earthindexembeddings/logo.png)
# Amazon Mining Watch

[![CC-4 license](https://img.shields.io/badge/License-CC--4-blue.svg)](https://creativecommons.org/licenses/by/4.0/)
[![Sensor](https://img.shields.io/badge/Sensor%F0%9F%9B%B0%EF%B8%8F-Sentinel2-blue)](https://www.esa.int/Applications/Observing_the_Earth/Copernicus/Sentinel-2)
[![Format](https://img.shields.io/badge/Format-GeoJSON%20%F0%9F%8C%8E%EF%B8%8F-blue)](https://geojson.org/)

This repository contains automated detections of artisanal gold mine
scars in Sentinel-2 satellite imagery.

The data is licensed under the [creative commons 4.0](https://creativecommons.org/licenses/by/4.0/) international license which, in summary, only requires attribution.

Code, description of the machine-learning models that generated the
detections, and additional versions of this data are available at the
[Amazon Mining Watch Github repository](https://github.com/earthrise-media/mining-detector).

### Data

Data are presented yearly from 2018-2024 and quarterly starting from
the second quarter of 2025. Each file represents cumulative detections of mine scars
begining in 2018. Detections are made on overlapping 480m x 480m
patches of the Earth's surface, merged into polygons where they overlap, and saved as
GeoJSON.

### Models

- Beginning with the 2024 data year, the model is an ensemble of
  classification heads trained atop embeddings from the [SSL4EO
  ViT-DINO/S16 geospatial foundation model](github.com/zhu-xlab/SSL4EO-S12),
  with ex-post-facto data cleaning by human review of satellite imagery.

- Data years 2018-2023 use a legacy ensemble of end-to-end trained
  convolutional neural networks.

### Data notes

- The break in models means that trends are not reliable from year
  2023 to 2024. In particular, the newer models are significantly more
  sensitive the the older, and the additional mining detected in
  2024 is partly an artefact of the model shift.

- The models producing this data are under active development. Data updates and
pixelwise segmentations of mining scars are expected in early 2026.

### Contact
Using this? Want work together? Ping us @ info@earthgenome.org or info@earthindex.ai

providers: role [processor] for 'Earthrise Media' is an AI guess — Assumed processor: hosts the upstream mining-detector GitHub repo where the ML model code lives. Verify before flipping to _state: reviewed.


Earth Index Embeddings

Global-coverage machine learning embeddings derived from Sentinel-2 satellite temporal mosaics using the SoftCon Model. Each embedding is a 384-dimensional vector capturing Earth surface characteristics at specific UTM grid locations and time periods.

remote-sensing sentinel-2 embeddings machine-learning earth-observation

Dataset files (17722 x .parquet)

Source Cooperative product page

## README.md

![EarthGenome](https://data.source.coop/earthgenome/earthindexembeddings/logo.png)
# EarthIndex Embeddings from Sentinel2 Temporal Mosaics

[![CC-4 license](https://img.shields.io/badge/License-CC--4-blue.svg)](https://creativecommons.org/licenses/by/4.0/)
[![Sensor](https://img.shields.io/badge/Sensor%F0%9F%9B%B0%EF%B8%8F-Sentinel2-blue)](https://www.esa.int/Applications/Observing_the_Earth/Copernicus/Sentinel-2)
[![Format](https://img.shields.io/badge/Format-GeoParquet%20%F0%9F%8C%8E%EF%B8%8F-blue)](https://geoparquet.org/)

This repository holds embeddings generated the entire globe for the given date ranges.

This data is licensed under the [creative commons 4.0](https://creativecommons.org/licenses/by/4.0/) international license which, in summary, only requires attribution.

### Model
These embeddings are generated from the [SoftCon Model](https://github.com/zhu-xlab/softcon/tree/main) published by [Zhu Labs](https://www.asg.ed.tum.de/sipeo/home/)

### Naming
Artifacts are name according to their original UTM grid number, concatenated with the start and end dates.  For example `31TDF_2024-01-01_2025-01-01.parquet` will give you embeddings for Barcelona (31TDF) over the course of 2024.

### Artifacts and Metadata
All assets are Geoparquet in EPSG:4626 projection with point geometries. The fields are as follows:
- id: uint64
- embedding: float[384]
- geometry: Point in WKB

### Contact
Using this? Want work together? Ping us @ info@earthgenome.org or info@earthindex.ai


Earth Index Imagery

Global collection of cloudless Sentinel-2 yearly imagery mosaics, covering all spectral bands except B10 at their native resolution. Assets are Cloud-Optimized GeoTIFFs (COGs) in Web Mercator (EPSG:3857) projection, organized by UTM grid tile and annual date range. Each asset includes quality metadata (good pixel percentage) and provenance references to the contributing Sentinel-2 L2A scenes.

remote-sensing satellite-imagery sentinel-2 multispectral mosaics

Dataset files (1045277 x .tif)

Source Cooperative product page

## README.md

![EarthGenome](https://data.source.coop/earthgenome/earthindexembeddings/logo.png)
# Earth Index yearly imagery mosaics

[![CC-4 license](https://img.shields.io/badge/License-CC--4-blue.svg)](https://creativecommons.org/licenses/by/4.0/)
[![Sensor](https://img.shields.io/badge/Sensor%F0%9F%9B%B0%EF%B8%8F-Sentinel2-blue)](https://www.esa.int/Applications/Observing_the_Earth/Copernicus/Sentinel-2)
[![Format](https://img.shields.io/badge/Format-COG%20%E2%9A%99%EF%B8%8F-purple)](https://www.cogeo.org/)

This repository holds a large (and growing!) collection of cloudless Senitnel 2 data. All bands (except B10 as this was generated from L2A) are available at their original native resolution.

This data is licensed under the creative commons 4.0 international license which, in summary, only requires attribution.

You can read more about how this was created [here](https://medium.com/earthrisemedia/earth-index-imagery-v2-40674f18b4d2).

### Resources
- [Earth Genome STAC](https://stac.earthgenome.org/collections/sentinel2-yearly-mosaics) instance for searching this collection
- [STAC Browser](https://browser.stac.earthgenome.org/collections/sentinel2-yearly-mosaics), human friendly interface

### Naming
Artifacts are name according to their original UTM grid number, concatenated with the start and end dates.  For example the `31TDF_2024-01-01_2025-01-01/` directory will give you bands over for Barcelona (31TDF) over the course of 2024.

### Artifacts and Metadata
All assets are COG encoded GeoTiffs in Web Mercator (EPSG:3857) projection. Besides the item id itself, the STAC catalog has the original UTM infornmation so you can calculate and then reproject to the original EPSG code, if needed.

The STAC assets also have 2 properties to help better understand the provience and quality of the images:

- `good_pxl_pct` is a the estimated percentage of "good" (non masked) pixels
- `generated_from` is an array of references to the Sentinel L2A assets which contributed to this asset

### Contact
Using this? Want work together? Ping us @ info@earthgenome.org or info@earthindex.ai

providers: role [producer, processor] for 'Earth Genome' is an AI guess — Credited as dataset owner/publisher; README contact and STAC catalog are earthgenome.org, but the processing pipeline is described under the Earth Index brand. Verify before flipping to _state: reviewed.

providers: role [producer] for 'European Space Agency (ESA)' is an AI guess — Sentinel-2 L2A imagery is the upstream source; ESA is not explicitly named as provider in the README but is the originating satellite operator. Verify before flipping to _state: reviewed.


Global Food Twin

The Global Food Twin is a model that estimates the most likely multi-modal transportation routes for food flows between producing and consuming subnational regions worldwide. It integrates food balance sheets, production maps, bilateral trade data, and transportation network optimization across 82 food groups and 3,787 subnational administrative regions. The dataset includes model inputs (downscaled trade allocations, infrastructure nodes, transport costs, production and demand rasters) and outputs (routed land, sea, and within-country food flow paths with mode and segment detail).

agriculture food-systems food-security trade-flows digital-twin

BEL CHN CZE DEU DNK ESP FRA HUN LVA NLD POL SVK SVN

ARG AUT BEL BGD BGR BLR BOL BRA CAF CAN CHE CHN COD COG COL CZE DEU FIN FRA HRV HUN IND ITA KHM LTU LUX MDA NLD PER POL PRT PRY ROU RUS SVK THA UKR USA VEN VNM

Alcoholic Beverages
Animal Fats
Aquatic Products, Other
Cereals - Excluding Beer
Eggs
Fish, Seafood
Fruits - Excluding Wine
Meat
Milk - Excluding Butter
Offals
Oilcrops
Pulses
Spices
Starchy Roots
Stimulants
Sugar Crops
Sweetners
Treenuts
Vegetables
Vegetable Oils

Dairy and Eggs
Fruits
Grains
Meat and Fish
Oils and Oilseed
Other
Pulses
Starchy Roots
Treenuts
Vegetables

[admin_code] ISO3 country code identifying the subnational demand region's country

[admin_code] ISO3 code of the reporting (exporting) country

[admin_code] ISO3 code of the partner (importing) country

Sugar beet.csv

IWWPorts_infrastructure.csv

lon

[latlon] Longitude of infrastructure node

lat

[latlon] Latitude of infrastructure node (port, rail junction, etc.)

fg1_fg2_fg3_lookup.csv

global_IWW_transport_cost.csv

[admin_code] ISO3 country code of the route origin node

[admin_code] ISO3 country code of the route destination node

fg1_totals_production_exports_calories_protein_iron_vita.csv

Flows_Yams.csv

[admin_code] Subnational admin region ID of flow origin (e.g. 'FRA.10_1')

[admin_code] Subnational admin region ID of flow destination

Flows_Sugar beet.csv

Flows_Sugar beet.csv

Flows_Sugar beet.csv

Flows_Fish__Body_Oil.csv

Source Cooperative product page

## README.md
# Global Food Twin
The Global Food Twin is a model that estimates the most likely transportation routes for food flows between producing and consuming regions worldwide. By combining food balance sheets, production maps, trade data, transportation networks, and optimization algorithms, we've created the best guess of how food moves across the globe—from producing to consuming regions—accounting for the complex multi-modal transportation systems. Visit a proof of concept [application visualizing this model and global food flows here](https://foodtwin.theplotline.org/).

A few facts about the Global Food Twin:
- Comprehensive coverage: 82 food groups across 3,787 subnational administrative regions globally
- Multi-modal transport: Maritime, land-based (rail, road, inland waterway), and combined transportation routes
- Detailed routing: Actual geospatial routes including specific ports, rail lines, and road networks

## Acknowledgements
This project was led by [Cameron Kruse](https://www.linkedin.com/in/krusecameron/) at [Earth Genome](https://www.earthgenome.org/) working with [Zia Mehrabi](https://www.colorado.edu/envs/zia-mehrabi), [Ginni Braich](https://www.linkedin.com/in/ginni-braich-41672337/), and [Kushank Bajaj](https://www.linkedin.com/in/kushank-bajaj-ph-d-536a8350/) [Better Planet Laboratory](https://betterplanetlab.com/) on the model; [Development Seed](https://developmentseed.org/) on the front end application; the model built on prior work by [Jasper Verschuur](https://www.linkedin.com/in/jasper-verschuur-076092104/) Assistant Professor at Delft University.

## Methods
This repository contains the data produced by the model and used by the Global Food Twin Application. More details about how this data was developed including the code used can be found in this repository [GitHub repository](https://github.com/Better-Planet-Laboratory/globalfoodtwin).

## the Data
This repository is organized into inputs and outputs. Below is a diagram of the folder structure. the [all_files.md](https://source.coop/earthgenome/foodtwin/all_files.md) document contains a full file structure.

```
├── 📁 inputs/
│   ├── 📁 downscaled_exports_imports/
│   │   ├── 📁 demand_allocated/
│   │   └── 📁 supply_allocated/
│   │       └── 📁 foodgroup1/
│   ├── 📁 infrastructure/
│   ├── 📁 production_demand/
│   │   ├── 📁 demand_rasters/
│   │   └── 📁 production_rasters/
│   └── 📁 transport_cost/
│       ├── 📁 land_transport/
│       └── 📁 maritime_transport/
└── 📁 outputs/
    ├── 📁 additional_files/
    ├── 📁 land/
    │   ├── 📁 land_domestic/
    │   └── 📁 land_reexports/
    ├── 📁 sea/
    │   ├── 📁 sea_domestic/
    │   └── 📁 sea_reexports/
    └── 📁 within_country/
```

## Using this data
This data is licensed under the [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/).

If you use this data in accordance with the [CC 4.0 license](http://creativecommons.org/licenses/by/4.0/), please cite:
`Better Planet Laboratory and Earth Genome. (2025). Global Food Twin. Harvard Dataverse. https://doi.org/10.7910/DVN/BLVPFU`

Reach out to [Cameron Kruse](https://www.linkedin.com/in/krusecameron/) with questions.

extent.spatial.bbox: omitted; spatial reference is categorical via [inputs/infrastructure.lat (latlon), inputs/infrastructure.lon (latlon), inputs/downscaled_exports_imports/demand_allocated.iso3 (admin_code), inputs/downscaled_exports_imports/demand_allocated.reporter_ISO3 (admin_code), inputs/downscaled_exports_imports/demand_allocated.partner_iso3 (admin_code), inputs/transport_cost/land_transport.from_iso3 (admin_code), inputs/transport_cost/land_transport.to_iso3 (admin_code), outputs/land/land_domestic.from_id_admin (admin_code), outputs/land/land_domestic.to_id_admin (admin_code)]. Add bbox once the placename → bbox gazetteer covers it.


Sentinel 2 Embeddings

No Description Provided


Sentinel 2 Cloud Free Temporal Mosaics

A large and growing collection of cloud-free Sentinel-2 temporal mosaics covering all spectral bands except B10, generated from L2A data at native resolution. Assets are Cloud-Optimized GeoTIFFs in Web Mercator projection, named by UTM grid tile and composite date range. Each asset includes provenance metadata tracking contributing source scenes and estimated good-pixel percentage.

remote-sensing sentinel-2 satellite-imagery temporal-mosaic multispectral cog

Dataset files (576453 x .tif)

Source Cooperative product page

## README.md

![EarthGenome](https://assets-global.website-files.com/642c1e823b4d441963f3fe5e/6604748ad5ef1e8e7c26dde2_Earth%20Genome%20Logo%20(Original).svg)
# Sentinel 2 Cloud Free Temporal Mosaics

[![CC-4 license](https://img.shields.io/badge/License-CC--4-blue.svg)](https://creativecommons.org/licenses/by/4.0/)
[![Sensor](https://img.shields.io/badge/Sensor%F0%9F%9B%B0%EF%B8%8F-Sentinel2-blue)](https://www.esa.int/Applications/Observing_the_Earth/Copernicus/Sentinel-2)
[![Format](https://img.shields.io/badge/Format-COG%20%E2%9A%99%EF%B8%8F-purple)](https://www.cogeo.org/)

This repository holds a large (and growing!) collection of cloudless Senitnel 2 data.  All bands (except B10 as this was generated from L2A) are available at their original native resolution.

This data is licensed under the [creative commons 4.0](https://creativecommons.org/licenses/by/4.0/) international license which, in summary, only requires attribution.

### Resources
- Earth Genome [STAC](https://stac.earthgenome.org) instance for searching this collection
- [STAC Browser](https://browser.stac.earthgenome.org/collections/sentinel2-temporal-mosaics), human friendly interface
- [Tile Cache, levels 0-8 (2.5gb)](https://data.source.coop/earthgenome/sentinel2-temporal-mosaics/cache.tar.gz) - if you wanted to set up a tile server, this archive has tile levels 0-8 pre-tiled.

### Naming
Artifacts are name accoring to their original UTM grid number, concatonated with the start and end dates.  For example `31TDF_2023-01-01_2024-01-01` will give you Barcelona (31TDF) over the course of 2023.

### Artifacts and Metadata
All assets are COG encoded GeoTiffs in Web Mercator (EPSG:3857) projection. Besides the item id itself, the STAC catalog has the original UTM infornmation so you can calculate and then reproject to the original EPSG code, if needed.

The STAC assets also have 2 properties to help better understand the provience and quality of the images:
- `good_pxl_pct` is a the estimated percentage of "good" (non masked) pixels
- `generated_from` is an array of references to the Sentinel L2A assets which contributed to this asset

### Contact
Using this? Want work together? Ping us @ info@earthgenome.org

providers: role [producer] for 'ESA / Copernicus' is an AI guess — ESA/Copernicus produces the underlying Sentinel-2 L2A source imagery; role inferred from data lineage, not stated in this README.. Verify before flipping to _state: reviewed.


Storm Events Database

The Storm Events Database documents storms and other weather phenomena having sufficient intensity to cause loss of life, injuries, property damage, and/or disruption to commerce across the United States. Records date from 1950 up to the current year. Created by the National Weather Service and the National Centers for Environmental Information, the dataset is presented here by The Internet of Water Coalition, The Commons, and Earth Genome.


Repository of all forecasts submitted to the Ecological Forecasting Initiative

A partitioned Parquet database of all forecasts submitted to the Ecological Forecasting Initiative's NEON Forecasting Challenge. It contains probabilistic ecological predictions across multiple themes (aquatics, terrestrial, phenology, etc.) from numerous research groups, partitioned by theme and model. The canonical source is data.ecoforecast.org; this copy is in development.

ecology biodiversity ecological-forecasting neon environmental-monitoring aquatics hydrology time-series environment

part-0.parquet

[admin_code] NEON site identifier (4-letter code) indicating the ecological monitoring station the forecast targets

Source Cooperative product page

## README.md
# Forecasts submitted to the Ecological Forecasting Initiative's NEON Challenge

***In development, experimental***

*At this time, canonical sources are still hosted on* `data.ecoforecast.org`.

## Resources

- [STAC Catalog](https://radiantearth.github.io/stac-browser/#/external/raw.githubusercontent.com/eco4cast/neon4cast-catalog/main/stac/catalog.json)

## Quickstart

Arrow provides an easy way to access remote parquet files from most languages widely used in data science.  Here we access all forecasts submitted to a particular theme. (Users looking to load only a single model should specify that on the path for faster access.  The STAC catalog can be used to explore available models).

The examples below show 'cloud-native' connections to the data -- 'lazy' connections that do not download the entire asset, but allow us to filter, subset, and operate directly on the remote data product.

### R Access

```{r}
library(arrow)
base = "s3://anonymous@us-west-2.opendata.source.coop"
repo = "eco4cast/neon4cast-forecasts"
theme = "aquatics"
uri = glue::glue("{base}/{repo}/parquet/{theme}?region=us-west-2")

open_dataset(uri)

```

### Python Access

```{python}
import pyarrow.dataset as ds

base = "s3://anonymous@us-west-2.opendata.source.coop"
repo = "eco4cast/neon4cast-forecasts"
theme = "aquatics"
uri = f"{base}/{repo}/parquet/{theme}?region=us-west-2"

ds.dataset(uri, format="parquet")
```

### duckdb

At this time, `duckdb` access substantially faster than `arrow`.

### R + duckdb

R users can get a dplyr-compatible lazy remote tibble as follows:

```{r}
# remotes::install_github("cboettig/duckdbfs")
library(duckdbfs)

base = "s3://anonymous@us-west-2.opendata.source.coop"
repo = "eco4cast/neon4cast-forecasts"
theme = "aquatics"
uri = glue::glue("{base}/{repo}/parquet/{theme}?region=us-west-2")

df = open_dataset(uri)
```

### Python + duckdb

[ibis](https://ibis-project.org/) provides a more Pythonic interface to SQL:

```{python}
import ibis
con = ibis.duckdb.connect()

base = "s3://us-west-2.opendata.source.coop"
repo = "eco4cast/neon4cast-forecasts"
theme = "aquatics"
uri = f"{base}/{repo}/parquet/{theme}/**"

con.raw_sql(f"""
INSTALL httpfs;
LOAD httpfs;
SET s3_region='us-west-2';
""")

db = con.read_parquet(uri)
```

providers: role [producer, host] for 'Ecological Forecasting Initiative' is an AI guess — EFI runs the challenge and aggregates submitted forecasts at data.ecoforecast.org; treating them as producer and original host. Verify before flipping to _state: reviewed.

providers: role [producer] for 'NEON (National Ecological Observatory Network)' is an AI guess — NEON provides the observation sites and targets that forecasts are issued against; role inferred from challenge structure. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [part-0.site_id (admin_code)]. Add bbox once a placename → bbox gazetteer is available.


Ecological Forecasting Initiative Scored Forecast Database

A partitioned Parquet database of ecological forecasts, observations, and probabilistic skill scores submitted to the Ecological Forecasting Initiative (EFI). Covers multiple ecological domains including aquatics, terrestrial ecosystems, beetles, and ticks, with data collected at NEON sites. Each record includes forecast distributions (mean, median, quantiles) and scoring metrics (CRPS, log score) for evaluating forecast accuracy.

ecology biodiversity timeseries neon environmental-monitoring aquatics hydrology terrestrial

part-0.parquet

[placename] NEON site identifier indicating the ecological monitoring location

Source Cooperative product page

license: AI returned 'unknown' (README does not state a license). Verify with publisher before flipping to _state: reviewed.

providers: role [producer, licensor] for 'Ecological Forecasting Initiative' is an AI guess — Assumed producer and licensor as the named initiative behind the forecast database; not explicitly stated in README.. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [part-0.site_id (placename)]. Add bbox once a placename → bbox gazetteer is available.


Canada's Energy Future 2023 (EF2023) is the Canada Energy Regulator's long-term outlook of national energy supply and demand projections from 2000 to 2050, covering electricity, crude oil, natural gas, hydrogen, and other energy carriers. It models three scenarios—Current Measures, Canada Net-zero, and Global Net-zero—with the latter two representing pathways to net-zero GHG emissions by 2050. Data are disaggregated by province/territory and energy type across 20 CSV files.

energy energy-projections electricity-generation crude-oil greenhouse-gas-emissions net-zero

Canada Net-zero
Current Measures
Global Net-zero

Canadian Light Sweet (CLS) - 2022 US$/bbl

Nova Inventory Transfer (NIT) - 2022 US$/MMBtu

West Texas Intermediate (WTI) - 2022 US$/bbl

Western Canadian Select (WCS) - 2022 US$/bbl

Exports from westcoast BC to International

(Upgraded Bitumen)
C5+
Conventional Heavy
Conventional Light
Field Condensate
In Situ Bitumen
Mined Bitumen
Total

Biomass / Geothermal
Coal & Coke
Hydro / Wave / Tidal
Natural Gas
Oil
Solar
Uranium
Wind

Battery Storage
Bioenergy
Bioenergy with CCUS
Coal and Coke
Coal with CCUS
Hydro
Hydrogen
Natural Gas
Natural Gas with CCUS
Offshore Wind
Oil
Onshore Wind
Solar (Distributed)
Solar (Utility scale)
Total
Uranium
Uranium SMR

Exports
Imports
Interprovincial In-Flows
Interprovincial Out-Flows
Net Exports
Net Interprovincial Out-Flows

Aviation Fuel
Biofuels
Biofuels & Emerging Energy
Diesel
Electricity
Heavy Fuel Oil
Hydrogen
LPG
Lubricants
Motor Gasoline
Natural Gas
Other
RPP
Total End-Use

Diesel
Electricity
Gasoline
Natural Gas
Oil

AB Petrochemical Demand
Ethane Not Recovered
Exports
Imports to AB from U.S.
Imports to ON from U.S.
ON Petrochemical Demand
Oil Sands Solvent Demand
Production from Gas Processing
Production from Oil Sands off-gas
Total Demand
Total Disposition
Total Production
Total Supply
Total Supply - Total Disposition

Agriculture
Buildings
Direct Air Capture
Electricity
Heavy Industry
Land Use, Land Use Change and Forestry
Low or Non-emitting Hydrogen Production
Oil and Gas
Total
Transportation
Waste and Others

Biomass
Electrolysis
Natural gas with CCS

Gross Domestic Product Deflator (2012=100)

Real Gross Domestic Product ($2012 Millions)

Coalbed Methane
Conventional
Conventional (Tight Portion)
Shale
Total

Coalbed Methane
Non Associated
Shale
Solution
Tight
Total

Production from Gas Processing - Pentanes Plus

Coal, Coke and Coke Oven Gas
Hydro
Natural Gas
Nuclear
Other Renewables and Landfill Gas
Refined Petroleum Products
Total

Thousand Barrels per day
Thousand Cubic Metres per day

Billion Cubic Feet per day
Million Cubic Metres per day

Alberta
British Columbia
Canada
Manitoba
Newfoundland and Labrador
Northwest Territories
Nova Scotia
Ontario
Saskatchewan

Alberta
British Columbia
Canada
Manitoba
New Brunswick
Newfoundland and Labrador
Northwest Territories
Nova Scotia
Nunavut
Ontario
Prince Edward Island
Quebec
Saskatchewan
Yukon

Alberta
British Columbia
Canada
Manitoba
New Brunswick
Newfoundland and Labrador
Nova Scotia
Ontario
Prince Edward Island
Quebec
Saskatchewan

Alberta
British Columbia
Manitoba
New Brunswick
Newfoundland and Labrador
Northwest Territories
Nova Scotia
Nunavut
Ontario
Prince Edward Island
Quebec
Saskatchewan
Yukon

Alberta
British Columbia
Saskatchewan
Western Canada

Alberta
British Columbia
Canada
New Brunswick
Northwest Territories
Nova Scotia
Ontario
Saskatchewan
Western Canadian Sedimentary Basin
Yukon

Commercial
Industrial
Residential
Total End-Use
Transportation

Commercial Industrial Residential Transportation

benchmark-prices-2023.csv

butane-2023.csv

crude-oil-production-2023.csv

Region

[placename] Canadian province or national aggregate

ef2023-data-dictionary.csv

Description of Data Value or Column Name

electricity-capacity-2023.csv

[placename] Canadian province/territory or national aggregate

electricity-capacity-technology-2023.csv

electricity-generation-2023.csv

electricity-generation-technology-2023.csv

electricity-interchange-2023.csv

end-use-demand-2023.csv

end-use-prices-2023.csv

ethane-2023.csv

greenhouse-gas-emissions-2023.csv

hydrogen-production-2023.csv

macro-indicators-2023.csv

[placename] National aggregate (Canada only)

natural-gas-drilling-2023.csv

[placename] Western Canadian province or regional aggregate

natural-gas-production-2023.csv

pentanes-plus-2023.csv

primary-demand-2023.csv

propane-2023.csv

Source Cooperative product page

## README.md
# canada_energy_supply_and_demand
Canada’s Energy Future 2023: Energy Supply and Demand Projections to 2050 (EF2023) is the latest long-term energy outlook from the Canada Energy Regulator (CER). The Canada’s Energy Future series explores how possible energy futures might unfold for Canadians over the long term. We use economic and energy models to explore how supply and demand for energy could evolve. EF2023 is the CER’s first long-term outlook that models scenarios where Canada reaches net-zero greenhouse gas (GHG) emissions by 2050. To model net-zero we begin with the end goal in mind: net-zero GHG emissions in 2050, and use our models to identify pathways to that point. This is a different approach compared to past versions of the report where we ran our models without restrictions, giving us insights into what a given premise meant for the future. EF2023 contains three scenarios. Two of these scenarios explore pathways where Canada achieves net-zero emissions by 2050. In the Global Net-zero Scenario, we assume Canada achieves net-zero emissions by 2050. We also assume the rest of the world reduces emissions enough to limit global warming to 1.5 Celsius (°C). In the Canada Net-zero Scenario, Canada also achieves net-zero emissions by 2050 but the rest of the world moves more slowly to reduce GHG emissions. The third scenario, the Current Measures Scenario, assumes limited action to reduce GHG emissions beyond measures in place today. In this scenario, we do not require our modeling results achieve net-zero GHG emissions in Canada by 2050. We also assume limited future global climate action.

extent.spatial.bbox: omitted; spatial reference is categorical via [crude-oil-production-2023.Region (placename), electricity-capacity-2023.Region (placename), electricity-capacity-technology-2023.Region (placename), electricity-generation-2023.Region (placename), electricity-generation-technology-2023.Region (placename), electricity-interchange-2023.Region (placename), end-use-demand-2023.Region (placename), end-use-prices-2023.Region (placename), hydrogen-production-2023.Region (placename), macro-indicators-2023.Region (placename), natural-gas-drilling-2023.Region (placename), natural-gas-production-2023.Region (placename), primary-demand-2023.Region (placename)]. Add bbox once a placename → bbox gazetteer is available.


Canada Natural Gas - Imports and Exports

Monthly and annual records of Canada's natural gas imports and exports by border crossing port, collected by the Canada Energy Regulator (CER) from holders of export and import authorizations. The dataset covers approximately 40 ports connecting Canada to the United States from 1985 to present, reporting volumes, values, and prices in both Canadian and US units.

energy-trade imports-exports energy-regulation commodity-trade

Aden
Brunswick
Cardston
Cardston/Emerson/Niagara Falls
Chippawa
Cornwall
Corunna
Courtright
Coutts
East Hereford
Elmore
Emerson
Emerson/Monchy
Fort Frances
Highwater
Huntingdon
Huntingdon/Kingsgate
Iroquois
Kingsgate
Kitimat
Loomis
Monchy
Napierville
Niagara Falls
North Portal
Ojibway (Windsor)
Philipsburg
Rainy River
Reagan Field
Sarnia
Sarnia/Blue Water
Sarnia/Sault Ste. Marie
Sault Ste Marie
Sierra
Sprague
St Clair
St Stephen
Total
Willow Creek
Windsor (Ojibway)

April August December February January July June March May November October September

natural-gas-exports-and-imports-annual.csv

Port

[placename] Named border crossing port through which natural gas imports or exports flow between Canada and the US

Region

[placename] Broad Canadian geographic region (East, MidWest, West) aggregating multiple ports

natural-gas-exports-and-imports-data-dictionary.csv

natural-gas-exports-and-imports-monthly.csv

Source Cooperative product page

## README.md
# canada_natural_gas_imports_exports
Canada is the world’s fifth largest producer of natural gas and accounts for around five per cent of global production. Natural gas production in Canada is predominantly from the Western Canadian Sedimentary Basin in British Columbia, Alberta, and Saskatchewan. Canadian natural gas supply currently exceeds domestic consumption. Canada’s natural gas markets are heavily integrated with those of the United States and Canada exports its surplus natural gas to the U.S., while importing smaller amounts from the U.S. into Central Canada. The Canada Energy Regulator regulates the export and import of natural gas. Orders or licenses are required to export or import natural gas to and from Canada. Holders of these authorizations report monthly on their activities to the CER. Import and export activities are available by Port from 1985 to present. Data is delayed by approximately 2 months.

providers: role [producer, licensor] for 'Canada Energy Regulator (CER)' is an AI guess — CER is explicitly stated as the regulatory body that collects and publishes this data; licensor role assumed as the authoritative government publishing body, but no explicit license is stated in the README.. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [natural-gas-exports-and-imports-annual.Port (placename), natural-gas-exports-and-imports-annual.Region (placename), natural-gas-exports-and-imports-monthly.Port (placename), natural-gas-exports-and-imports-monthly.Region (placename)]. Add bbox once the placename → bbox gazetteer covers it.