Cloud Native MGRS

A complete Military Grid Reference System (MGRS) 1-kilometer reference grid derived from National Geospatial-Intelligence Agency (NGA) source data, converted to cloud-native GeoParquet format. Contains over 305 million polygon grid cells spanning global extent from -80° to 84° latitude, each encoded with MGRS codes, Grid Zone Designators, and 10-kilometer tile identifiers. Data is available as a single file and as ~700 GZD-partitioned files for efficient spatial querying.

geoparquet global-coverage remote-sensing mgrs coordinate-system utm-grid grid-reference

full

GZD

[admin_code] MGRS Grid Zone Designator (6° × 8° zone code used as partition key)

MGRS

[spatial_index] Full MGRS grid reference code identifying the 1 km cell

[spatial_index] 10-kilometer MGRS tile identifier containing this cell

geometry

[geometry] Polygon boundary of each 1 km MGRS grid cell in EPSG:4326

2 files sampled for this table

gzd_partition

GZD

[admin_code] MGRS Grid Zone Designator; one partition file per GZD value

geometry

[geometry] Polygon boundary of each 1 km MGRS grid cell in OGC:CRS84

3 files sampled for this table

partition/by_kdtree

GZD

[admin_code] MGRS Grid Zone Designator

5 files sampled for this table

Source Cooperative product page

## README.md
## About

This project crawled the map download links at https://earth-info.nga.mil/index.php?dir=coordsys&action=utm-1km-polyline-dloads
and put them into a GeoParquet files that follow the [Best Practices for Distributing GeoParquet](https://github.com/opengeospatial/geoparquet/blob/main/format-specs/distributing-geoparquet.md). The `full` directory contains the complete dataset as a single GeoParquet file. For now it just contains `mgrs_clean.parquet` which is all the
download links that worked, but removing the ShapeLen, ShapeArea and area columns, as they weren't all that consistent.

## 📦 Dataset Overview

### **mgrs_clean.parquet**
**Size:** 5.87 GB
**Compression:** ZSTD
**Rows:** 305,799,745
**Row Groups:** 2,483
**GeoParquet Version:** 1.0.0
**CRS:** EPSG:4326
**Bounding Box:** `[-180.000000, -80.000000, 180.000000, 84.000193]`

#### Columns
| Name        | Type   |
|-------------|--------|
| EASTING     | string |
| NORTHING    | string |
| kmSQ_ID     | string |
| GZD         | string |
| MGRS        | string |
| MGRS_10km   | string |
| geometry 🌍 | binary |

---

### Partitioning Notes

The `partitioned/` directory currently includes **`gzd_partition/`**, containing ~700 partition files (one per GZD).
These partitions:

- originate from `mgrs_clean.parquet`
- include **GeoParquet 1.1 bounding boxes** for faster spatial filtering
- are intended to support future **VecOREL** extensions:
  - administrative division references
  - geometry metrics

---

## 📦 Sample Subset

### **mgrs_10X.parquet**
**Size:** 9.02 MB
**Compression:** ZSTD
**Rows:** 187,646
**Row Groups:** 1
**GeoParquet Version:** 1.1.0
**CRS:** EPSG:4326
**Bounding Box:** `[-180.000000, -80.000000, 180.000000, 84.000193]`

#### Columns
| Name        | Type |
|------------|------|
| EASTING     | string |
| NORTHING    | string |
| kmSQ_ID     | string |
| GZD         | string |
| MGRS        | string |
| MGRS_10km   | string |
| geometry 🌍 | binary |
| bbox | struct `<xmin: double, ymin: double, xmax: double, ymax: double>` |

---

## Usage

The best way to use these is to treat the partition as a full parquet dataset and query just what you need out of it. With DuckDB
you can do:

```
CREATE TABLE mgrs_grid AS (
  SELECT * FROM read_parquet('s3://us-west-2.opendata.source.coop/tge-labs/mgrs/gzd_partition/*/*.parquet')
           WHERE "bbox".xmin BETWEEN 11.14686 AND 14.853739
           AND "bbox".ymin BETWEEN 50.49916 AND 54.37598);
```

And then you'll have the table locally. You can go straight to a parquet file with

```
COPY (
  SELECT * FROM read_parquet('s3://us-west-2.opendata.source.coop/tge-labs/mgrs/gzd_partition/*/*.parquet')
           WHERE "bbox".xmin BETWEEN 11.14686 AND 14.853739
           AND "bbox".ymin BETWEEN 50.49916 AND 54.37598)
    TO 'my_mgrs_grid.parquet' (FORMAT PARQUET, COMPRESSION zstd, COMPRESSION_LEVEL 15);
```

extent.spatial.bbox: omitted; spatial reference is categorical via [full.geometry (geometry), full.MGRS (spatial_index), full.MGRS_10km (spatial_index), full.GZD (admin_code), gzd_partition.geometry (geometry), gzd_partition.MGRS (spatial_index), gzd_partition.MGRS_10km (spatial_index), gzd_partition.GZD (admin_code), partition/by_kdtree.geometry (geometry), partition/by_kdtree.MGRS (spatial_index), partition/by_kdtree.MGRS_10km (spatial_index), partition/by_kdtree.GZD (admin_code)]. Add bbox once the placename → bbox gazetteer covers it.


OpenBuildingMap

Global dataset of 2.7 billion building footprints with associated metadata including height, usage classification, and floorspace. Covers all terrestrial areas worldwide.

building-footprints building-heights building-usage floor-area

Dataset files (1270 x .parquet)

Source Cooperative product page


TerraBit


Global Risk/Resilience Index - Supporting Data

A collection of global, open datasets relating to climate hazards and infrastructure systems, processed into analysis-ready formats. Hazards include: coastal and river flooding, extreme heat and drought, and tropical cyclone wind speeds. Exposure layers include: population, built-up area, roads, railways, power plants and transmission lines. Contextual information includes: elevation, land use and land cover, and administrative boundaries.


Trazo Fields

Agricultural field boundaries produced by Trazo models for key geographies in South America for the 2023-2024 planting season. Post-processed.


Trazo Training Data

# South America Field Boundary Annotations

A training dataset of **46,908** high-quality, diverse field boundary annotations with representation from 17 ecoregions and a test set of **7,453** field boundary annotations.


HURREVAC Storm Advisory Archive

This dataset provides a point-in-time archive and persistent mirror of the tropical cyclone advisory data from HURREVAC, the decision support tool used by the National Hurricane Program (a partnership between FEMA, USACE, and NOAA).

The data was programmatically captured from the HURREVAC API (/v1/advisories/storms) to ensure long-term accessibility for the emergency management and research communities, specifically to mitigate the risk of data loss due to potential funding shifts or service changes.

The archive consists of JSON-formatted records containing:

Current Storm Positions: Historical and active tropical cyclone coordinates.

Forecast Tracks: Official forecast data as issued during active storm events.

Metadata: Unique identifiers and structural data used by the Sea Island Software HVX platform.

Data Source
Provider: National Hurricane Program via HURREVAC API.
Original URL: http://hvx.hurrevac.com/hvx-api/v1/advisories/storms
Format: JSON


Global Administrative Area Boundaries

Administrative boundaries for several levels (GADM Level 0 - 5)
Source https://gadm.org
License: https://gadm.org/license.html


maxar-opendata-v1.1

Testing out conversion of the Maxar Open Data Program data to geoparquet v1.1


Test Files

A variety of files for testing Source Cooperative.


Wall tiles

Uploading TIFF files


Test Title

Test description


USDA Cropland Data Layer

A raster crop-specific land cover data layer of the continental United States, published annually by the USDA National Agricultural Statistics Service.


testproduct

this is a test


USASpending Data

Bulk data from USASpending.gov


A merged global dataset combining Google's V3 Open Buildings (1.8 billion detections) and Microsoft's Global Building Footprints (1.24 billion detections) into 2.58 billion building footprints covering Africa, South Asia, South-East Asia, Latin America, and more. Each footprint carries its source label, confidence score, area in square meters, and country ISO code. Available in GeoParquet, FlatGeobuf, and PMTiles formats, partitioned by country and S2 grid.

building-footprints remote-sensing urban-mapping global-coverage geoparquet

AFG.fgb

AGO.fgb

ALB.fgb

AND.fgb

ARE.fgb

2211267417038913536.fgb

[spatial_index] S2 grid cell ID used for sub-country partitioning (S2-partitioned assets only)

[admin_code] 3-letter ISO country code for the footprint's country (S2-partitioned assets only)

geohash

[spatial_index] Geohash at precision 8 used for spatial ordering (S2-partitioned assets only)

2220274616293654528.fgb

2229281815548395520.fgb

2017612633061982208.fgb

Source Cooperative product page

## README.md
# Google-Microsoft Open Buildings - combined by VIDA
## Overview
This dataset merges Google's V3 Open Buildings and Microsoft's latest Building Footprints. It contains 2,579,035,323 footprints and is divided into 185 partitions. Each footprint is labelled with its respective source, either Google or Microsoft. It can be accessed in cloud-native geospatial formats such as GeoParquet, FlatGeobuf and PMTiles.

## See it in action
You can [Observable](https://observablehq.com/d/d84e078c16baa3ae) to get a quick overview of the dataset or go to [VIDA](https://map.vida.place/explore) to see it in action.

## Original datasets

### Google
The original Google V3 open buildings is downloadable from [this link](https://sites.research.google/open-buildings/#download) as gzipped CSV files. Here are some key details about the original dataset:

> The dataset contains 1.8 billion building detections, across an inference area of 58M km2 within Africa, South Asia, South-East Asia, Latin America and the Caribbean.
>
> Each building in the dataset has a polygon defining its footprint on the ground, a confidence score indicating how certain we are that this is a building, and a Plus Code corresponding to the centre of the building. There is no information about the type of building, its street address, or any details other than its geometry.

For more comprehensive information, please visit the [description page](https://sites.research.google/open-buildings/). You can also check out the [FAQ](https://sites.research.google/open-buildings/#faq) section for additional information.

### Microsoft
The latest version of Microsoft's building footprints can be downloaded from [Microsoft Planetary Computer](https://planetarycomputer.microsoft.com/dataset/ms-buildings) as gzipped partitioned files.

> The Microsoft Global Open Buildings dataset was generated through Bing Maps, which detected a total of 1.24 billion buildings. These buildings were identified using imagery from Bing Maps, encompassing data collected between 2014 and 2023, including images from Maxar, Airbus, and IGN France.
>

For more detailed information please visit the [github page](https://github.com/microsoft/GlobalMLBuildingFootprints/)

## Data Formats
The data is available in the following formats:
- GeoParquet 1.1.0
  - By country - single file
  - By country - S2 partitioned
- FlatGeobuf
  - By country - single file
  - By country - S2 partitioned
- PMTiles
  - Global - single layer
  - Global - layer per country based on the 3-letter ISO code
  - By country

## Partitioning
This extensive dataset is organized into 185 root partitions. Each partition typically corresponds to a country's administrative boundary, as defined by the Comprehensive Global Administrative Zones (CGAZ) at the ADM0 level, which can be accessed [here](https://www.geoboundaries.org/downloadCGAZ.html). There is also a sub-partition available, based on the S2 grid.

### By country
Both FlatGeobuf and GeoParquet are categorized by country boundaries, in accordance with the ADM0 level of the CGAZ geoboundary definition. This means that building footprints are separated by countries within each format. For naming conventions, we utilize the country's ISO CODE.

`/geoparquet/by_country/country_iso={ISO}/{ISO}.parquet`

> **Note:** There is a partition labeled `country_iso=None`, which represents a MULTIPOLYGON containing geoboundaries (POLYGONS) that have not been explicitly defined or named by CGAZ. These geoboundaries are still captured by CGAZ at the ADM0 level, but they lack specific names and therefore labelled `null`. As a result, building footprints located within these geoboundaries are included in this partition labeled `country_iso=None`. For instance, the area between Sudan and South Sudan includes a piece of land known as "Abyei" which remains unclaimed due to recurring conflicts, and therefore, it lacks an assigned name.

### By country + S2 grid
To enhance performance, particularly with GeoParquet files, we've introduced an S2 sub-partitioning strategy. Each ISO partition is further divided using an S2 grid ID, ensuring a cap of 20 million building footprints per grid ID. This S2 grid partitioning is exclusive to GeoParquet files.

`/geoparquet/by_country_s2/country_iso={ISO}/{S2_GRID_ID}.parquet`

## Schema
Each row in the dataset provides information on a specific building footprint with associated information on individual columns:
- **boundary_id (INTEGER):** A unique ID linking the CGAZ level 0 boundary ISO to an integer, created for partitioning the datasets within BigQuery.
- **confidence (FLOAT):** A metric denoting the model's confidence about the accuracy of the building footprint. Microsoft-sourced footprints set this column to null since the original dataset doesn't feature this attribute.
- **bf_source (STRING):** Indicates the footprint's origin - Google or Microsoft.
- **area_in_meters (FLOAT):** Represents the polygon's area in square meters.
- **s2_id (INT):** Exclusive to the S2 partitioning scheme, it represents the S2 grid ID.
- **country_iso (STRING):** 3-letter ISO code of the country the footprint belongs within.
- **geohash (STRING):** Geohash for the geometry at a precision level of 8.
- **bbox (STRUCT):** Struct containing `xmin, ymin, xmax, ymax` values for the bounding box of the geometry.

## Data Processing
We invite you to read our [blog post](https://cloudnativegeo.org/blog/2023/09/introducing-the-ultimate-cloud-native-building-footprints-dataset/) for more detailed information on our dataset merging approach, which includes insights into the optimization techniques we investigated and the query performance on BigQuery. In this section, we provide a high-level summary of the merging process, highlighting its crucial aspects.
\
We imported both datasets into BigQuery for further processing. From the Google dataset, we excluded columns like `full_plus_code`, `latitude`, and `longitude`. For the Microsoft dataset we did not drop any columns.
We then matched each building footprint with a boundary ID, determined by the intersection of its centroid with the country geoboundaries in the CGAZ ADM0 dataset. Footprints whose centroids didn't overlap with any country geoboundary were mapped to the nearest geoboundary based on their centroid's position.

## Contact details
If you'd like more information about the dataset or the processing steps, feel free to write an email to [support@vida.place](mailto:support@vida.place).

## Changelog
Current version: 2.0

### Version 2.0 - 2024-09-04
- Add 32,784,238 building footprints for various regions by updating to the latest Microsoft [GlobalMLBuildingFootprints](https://github.com/microsoft/GlobalMLBuildingFootprints) as of 2024-05-28.
- Update to GeoParquet schema version 1.1.0.
  - Includes bbox struct for easy filtering.
- Introduce spatial ordering by geohash for FlatGeobuf and GeoParquet files.
- Add PMTiles files per country.
- Add PMTiles file with a layer per country ISO code.

### Version 1.1 - 2023-10-02
- Added 11,631,283 building footprints for Morroco from the [Google Earthquake dataset](https://data.humdata.org/dataset/openbuildings_morocco_earthquake_footprint)
- Added 24,532 building footprints for Libya from the [Google Derna Flooding dataset](https://data.humdata.org/dataset/buildings-footprint-floods-derna-libya)
- Building footprints are added to the GeoParquet, FlatGeobuf and PMTiles archives.
- Fixed the missing GeoParquet version bug.
- Refactored S2 grid naming strategy from unsigned 64bit integers to signed 64bit integers.

### Version 1 - 2023-08-29

## Dataset Licenses
The data is shared under the [Open Data Commons Open Database License (ODbL) v1.0](https://opendatacommons.org/licenses/odbl/1-0/) license.

providers: role [processor, licensor] for 'VIDA' is an AI guess — VIDA performed the merge and publishes the combined dataset; licensor role inferred from their ODbL release — not explicitly stated. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [2211267417038913536.s2_id (spatial_index), 2211267417038913536.country_iso (admin_code), 2211267417038913536.geohash (spatial_index)]. Add bbox once a placename → bbox gazetteer is available.


A global dataset of 2.7 billion building footprints created by merging Google's V3 Open Buildings, Microsoft's GlobalMLFootprints, and OpenStreetMap building data. Each record includes the footprint polygon, source attribution (Google, Microsoft, or OSM), area in square meters, confidence score, and country ISO code. Data is partitioned by country and S2 grid, available in GeoParquet, FlatGeobuf, and PMTiles formats.

building-footprints remote-sensing openstreetmap administrative-boundaries global-coverage geoparquet

AFG.fgb

[spatial_index] S2 grid cell ID used for sub-country partitioning (cap of 20M footprints per cell)

[admin_code] ISO 3-letter country code assigned from CGAZ ADM0 boundaries

[spatial_index] Geohash spatial index for the building footprint centroid

AGO.fgb

ALB.fgb

AND.fgb

ARE.fgb

2211267417038913536.fgb

2220274616293654528.fgb

2229281815548395520.fgb

2017612633061982208.fgb

Source Cooperative product page

## README.md
# Google-Microsoft-OSM Open Buildings - combined by VIDA
## Overview
This dataset merges Google's V3 Open Buildings, Microsoft's GlobalMLFootprints, and OpenStreetMap building footprints. It contains 2,705,459,584 footprints and is divided into 200 partitions. Each footprint is labelled with its respective source, either Google, Microsoft, or OpenStreetMap. It can be accessed in cloud-native geospatial formats such as GeoParquet, FlatGeobuf and PMTiles.

## See it in action
You can [Observable](https://observablehq.com/d/d84e078c16baa3ae) to get a quick overview of the dataset or go to [VIDA](https://map.vida.place/explore) to see it in action.

## Original datasets

### Google
The original Google V3 open buildings is downloadable from [this link](https://sites.research.google/open-buildings/#download) as gzipped CSV files. Here are some key details about the original dataset:

> The dataset contains 1.8 billion building detections, across an inference area of 58M km2 within Africa, South Asia, South-East Asia, Latin America and the Caribbean.
>
> Each building in the dataset has a polygon defining its footprint on the ground, a confidence score indicating how certain we are that this is a building, and a Plus Code corresponding to the centre of the building. There is no information about the type of building, its street address, or any details other than its geometry.

For more comprehensive information, please visit the [description page](https://sites.research.google/open-buildings/). You can also check out the [FAQ](https://sites.research.google/open-buildings/#faq) section for additional information.

### Microsoft
The latest version of Microsoft's building footprints can be downloaded from [Microsoft Planetary Computer](https://planetarycomputer.microsoft.com/dataset/ms-buildings) as gzipped partitioned files.

> The Microsoft Global Open Buildings dataset was generated through Bing Maps, which detected a total of 1.24 billion buildings. These buildings were identified using imagery from Bing Maps, encompassing data collected between 2014 and 2023, including images from Maxar, Airbus, and IGN France.

For more detailed information please visit the [github page](https://github.com/microsoft/GlobalMLBuildingFootprints/)

### OpenStreetMap
The OpenStreetMap building footprints are sourced from [OpenStreetMap](https://www.openstreetmap.org/). The dataset is updated regularly and contains building footprints from all over the world.

> OpenStreetMap is a collaborative project that creates a free editable map of the world. The data is collected by volunteers and can be used for various purposes, including building footprints.
> There is more than 615 million building footprints in the OpenStreetMap dataset at the time of writing.

For more detailed information please visit the [OpenStreetMap Wiki](https://wiki.openstreetmap.org/wiki/Buildings)

## Data Formats
The data is available in the following formats:
- GeoParquet 1.1.0
  - By country - single file
  - By country - S2 partitioned
- FlatGeobuf
  - By country - single file
  - By country - S2 partitioned
- PMTiles
  - Global - single layer
  - Global - layer per country based on the 3-letter ISO code
  - By country

## Partitioning
This extensive dataset is organized into 182 root partitions. Each partition typically corresponds to a country's administrative boundary, as defined by the Comprehensive Global Administrative Zones (CGAZ) at the ADM0 level, which can be accessed [here](https://www.geoboundaries.org/downloadCGAZ.html). There is also a sub-partition available, based on the S2 grid.

### By country
Both FlatGeobuf and GeoParquet are categorized by country boundaries, in accordance with the ADM0 level of the CGAZ geoboundary definition. This means that building footprints are separated by countries within each format. For naming conventions, we utilize the country's ISO CODE.

`/geoparquet/by_country/country_iso={ISO}/{ISO}.parquet`

> **Note:** There is a partition labeled `country_iso=None`, which represents a MULTIPOLYGON containing geoboundaries (POLYGONS) that have not been explicitly defined or named by CGAZ. These geoboundaries are still captured by CGAZ at the ADM0 level, but they lack specific names and therefore labelled `null`. As a result, building footprints located within these geoboundaries are included in this partition labeled `country_iso=None`. For instance, the area between Sudan and South Sudan includes a piece of land known as "Abyei" which remains unclaimed due to recurring conflicts, and therefore, it lacks an assigned name.

### By country + S2 grid
To enhance performance, particularly with GeoParquet files, we've introduced an S2 sub-partitioning strategy. Each ISO partition is further divided using an S2 grid ID, ensuring a cap of 20 million building footprints per grid ID. This S2 grid partitioning is exclusive to GeoParquet files.

`/geoparquet/by_country_s2/country_iso={ISO}/{S2_GRID_ID}.parquet`

## Schema
Each row in the dataset provides information on a specific building footprint with associated information on individual columns:
- **boundary_id (INTEGER):** A unique ID linking the CGAZ level 0 boundary ISO to an integer, created for partitioning the datasets within BigQuery.
- **confidence (FLOAT):** A metric denoting the model's confidence about the accuracy of the building footprint. Microsoft-sourced footprints set this column to null since the original dataset doesn't feature this attribute.
- **bf_source (STRING):** Indicates the footprint's origin - Google or Microsoft.
- **area_in_meters (FLOAT):** Represents the polygon's area in square meters.
- **s2_id (INT):** Exclusive to the S2 partitioning scheme, it represents the S2 grid ID.

## Data Processing
We invite you to read our [blog post](https://medium.com/vida-engineering/updating-the-ultimate-cloud-native-building-footprints-dataset-6d4384cb93c4) for more detailed information on our dataset merging approach, which includes insights into the optimization techniques we investigated and the query performance on BigQuery. In this section, we provide a high-level summary of the merging process, highlighting its crucial aspects.
\
We imported both datasets into BigQuery for further processing. From the Google dataset, we excluded columns like `full_plus_code`, `latitude`, and `longitude`. For the Microsoft dataset we did not drop any columns.
We then matched each building footprint with a boundary ID, determined by the intersection of its centroid with the country geoboundaries in the CGAZ ADM0 dataset. Footprints whose centroids didn't overlap with any country geoboundary were mapped to the nearest geoboundary based on their centroid's position.

## Contact details
If you'd like more information about the dataset or the processing steps, feel free to write an email to [support@vida.place](mailto:support@vida.place).

## Changelog

Current version: 2.0

### Version 2.0 - 2024-09-04
- Initial release!
  - 2,705,459,584 building footprints for various regions by merging Google's V3 Open Buildings, Microsoft's GlobalMLFootprints, and OpenStreetMap building footprints.
  - Using GeoParquet schema version 1.1.0.
  - Data is partitioned by country and S2 grid.
  - Data is available in GeoParquet, FlatGeobuf, and PMTiles formats.
  - Google Open Buildings last updated 2023-10-02
  - Microsoft GlobalMLFootprints last updated 2024-05-28
  - OpenStreetMap last updated 2024-05-28

## Dataset Licenses
The data is shared under the [Open Data Commons Open Database License (ODbL) v1.0](https://opendatacommons.org/licenses/odbl/1-0/) license.

providers: role [processor, licensor] for 'VIDA' is an AI guess — Processor role is stated; licensor inferred as VIDA publishes the combined dataset under ODbL. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [AFG.country_iso (admin_code), AFG.geohash (spatial_index), AFG.s2_id (spatial_index)]. Add bbox once a placename → bbox gazetteer is available.


Merged Google-Microsoft Buildings

This dataset is a combination of Google's and Microsoft's Open Buildings. It is an openly accessible collection that provides the geometric information of buildings across a large area in Africa, South Asia, and South-East Asia. This specific version of the dataset is organized based on admin 0 boundaries and is accessible in cloud-native geospatial formats like PMTiles, FlatGeobuf, and GeoParquet.

geoparquet
building footprints
pmtiles
flatgeobuf


Global 100m resolution raster maps of the Terrestrial Human Footprint (HFP-100) for 2017–2021, quantifying cumulative anthropogenic pressure on land surfaces. Pressure layers include built environments, croplands, pasture, population density, nighttime lights, roads, railways, and navigable waterways, combined into a standardized 0–50 score. Produced annually to monitor changing human impacts on terrestrial ecosystems worldwide.

land-use remote-sensing human-footprint biodiversity anthropogenic-pressure population-density land-cover

Dataset files (5 x .tif)

Source Cooperative product page

## README.md
# Global 100m Terrestrial Human Footprint (HFP-100) v1.2

This repository contains the global results for the Human Footprint 100m product for 2017, 2018, 2019, 2020 and 2021. The Human Footprint (HFP) is used to monitor changing anthropogenic pressures on a yearly cadence based on land cover change, population density, nighttime lights, roads, railways, and navigable rivers.

This version 1.2 includes the years 2017-2021 based on land cover from the Esri/Impact Observatory 10m Annual Land Use Land Cover (9-class) dataset. Version 1 of this dataset covering the years 2015-2020 can be found at https://datadryad.org/stash/dataset/doi:10.5061/dryad.ttdz08m1f.

This dataset was created by combining data on human pressures across the period 2017-2021 to map: 1) Land cover change (built environments, crop lands, and pasture lands), 2) population density, 3) electric infrastructure, 4) roadways, 5) railways, and 6) navigable waterways. Each pressure layer is assigned a score relative to its level of human pressure, then computed into a standardized scale of 0-50 as the sum of all pressure layers. Pressures are not mutually exclusive, rather the co-occurrence of pressures is intended to identify the greatest levels of human impact. The majority of layers cover the complete time period of 2017-2021, however pressures from pasture, roads, and railways are treated as static due to limitations in the input datasets.

Overall methodology is based on the following:
 - B. A. Williams, O. Venter, J. R. Allan, S. C. Atkinson, J. A. Rehbein, M. Ward, M. Di Marco, H. S. Grantham, J. Ervin, S. J. Goetz, A. J. Hansen, P. Jantz, R. Pillay, S. Rodríguez-Buriticá, C. Supples, A. L. S. Virnig, J. E. M. Watson, Change in Terrestrial Human Footprint Drives Continued Loss of Intact Ecosystems. One Earth. 3, 371–382 (2020).
 - E. W. Sanderson, M. Jaiteh, M. A. Levy, K. H. Redford, A. V. Wannebo, G. Woolmer, The Human Footprint and the Last of the Wild: The human footprint is a global map of human influence on the land surface, which suggests that human beings are stewards of nature, whether we like it or not. BioScience. 52, 891–904 (2002).
 - O. Venter, E. W. Sanderson, A. Magrach, J. R. Allan, J. Beher, K. R. Jones, H. P. Possingham, W. F. Laurance, P. Wood, B. M. Fekete, M. A. Levy, J. E. M. Watson, Global terrestrial Human Footprint maps for 1993 and 2009. Sci. Data. 3, 160067 (2016).

Please see the following publication for more information: Gassert F., Venter O., Watson J.E.M., Brumby S.P., Mazzariello J.C., Atkinson S.C. and Hyde S., An Operational Approach to Near Real Time Global High Resolution Mapping of the Terrestrial Human Footprint. Front. Remote Sens. 4:1130896 doi: 10.3389/frsen.2023.1130896 (2023). https://www.frontiersin.org/articles/10.3389/frsen.2023.1130896/full

## Data information

HFP scores range from 0 to 50, with 0 representing virtually no human pressure and 50 representing severe human pressure on the environment. The data available here is multiplied by 1000 and converted to uint16 data type to reduce file sizes such that a value of 50000 corresponds to a score of 50. No data regions are coded as 65535.

## Code/Software

Data was produced using open source Python scripts and processed in the Microsoft Planetary Computer infrastructure. Scripts used to produce this data are available at https://gitlab.com/impactobservatory/dwi-humanfootprint.

## Suggested Citation

F. Gassert, O. Venter, J. E.M. Watson, S.P. Brumby, J.C. Mazzariello, S.C. Atkinson, S. Hyde. 2023: "Global 100m Terrestrial Human Footprint (HFP-100) v1.2"

## License

Creative Commons CC-BY-4.0

## Disclosure

Funding provided by Microsoft.

providers: role [producer, processor] for 'Impact Observatory' is an AI guess — Named as source of input land cover data and code repository host; assumed processor role based on GitLab repo under impactobservatory. Verify before flipping to _state: reviewed.

providers: role [producer] for 'Gassert et al. / study authors' is an AI guess — Corresponding authors of the methodology publication; assumed producers of the HFP-100 dataset. Verify before flipping to _state: reviewed.

providers: role [processor] for 'Microsoft' is an AI guess — Provided Planetary Computer infrastructure for processing; funding disclosed but no explicit role label given. Verify before flipping to _state: reviewed.


Global gridded raster data for estimating supply chain impacts related to deforestation, greenhouse gas emissions, natural ecosystem conversion, and biodiversity loss. Originally prepared for the LandGriffon service following international standards for reporting and target setting in the agriculture, forestry, and land use sector. Includes 100m-resolution impact layers and 1km-resolution annualized impact-per-hectare layers for five indicators across two time windows.

agriculture land-use deforestation conservation carbon-emissions biodiversity

Dataset files (11 x .tif)

Source Cooperative product page

## README.md
# Land, carbon and biodiversity data for supply chain impact calculation

This repository contains global gridded data for estimating supply chain impacts relating to deforestation, related greenhouse gas emissions, conversion of natural ecosystems, and loss of intact or high integrity ecosystems. These data were originally prepared for the LandGriffon service (https://landgriffon.com) following international standards and guidelines for reporting and target setting in the agriculture, forestry, and land use sector. In particular, they employ a spatial adaptation of the Statistical Land Use Change (sLUC) by land occupation accounting method in which responsibility for impacts are allocated across a 50km radius instead of across a jurisdiction.

Methodology and usage notes: https://www.biorxiv.org/content/10.1101/2023.11.01.565036v1

Code: https://github.com/Vizzuality/lg-landscape-indicators-processing

Five indicators are included, with two layers per indicator: the impact, and annualized impact per hectare of human land use.

**Impact layers (100m)**

| file | indicator | units |
| --- | --- | --- |
| deforest_100m_cog.tif | Deforestation (2002-2022) | portion of cell |
| deforest_carbon_100m_cog.tif | Deforestation carbon (2002-2022) | tC/ha |
| natcrop_expansion_100m_cog.tif | Cropland expansion in natural ecosystems (2020-2022) | portion of cell |
| natcrop_reduction_100m_cog.tif | Cropland reduction in natural ecosystems (2020-2022) | portion of cell |
| natcrop_bii_100m_cog.tif | Biodiversity Intactness score of converted areas | score |
| natcrop_fii_100m_cog.tif | Forest Landscape Integrity Index of converted areas | score |

**Annualized impact per hectare of human land use (1000m)**

| file | indicator | units |
| --- | --- | --- |
| deforest_by_human_lu_50km_1000m_cog.tif | Deforestation | ha/yr / haLU |
| deforest_carbon_by_human_lu_50km_1000m_cog.tif | Deforestation carbon emissions | tCO2/yr / haLU |
| natural_crop_net_conversion_by_human_lu_50km_1000m_cog.tif | Net cropland expansion in natural lands | ha/yr / haLU |
| natural_crop_bii_loss_by_human_lu_50km_1000m_cog.tif | Loss of forest integrity to cropland expansion | score / haLU |
| natural_crop_flii_loss_by_human_lu_50km_1000m_cog.tif | Loss of biodiversity intactness to cropland expansion | score / haLU |

Of note, these datasets are not constrained to areas of current human land use, e.g. such that land use in remote wilderness may be assigned a maximum impact score.

See https://landgriffon.com/methodology for techniques for combining these results with land footprint data and production data to calculate the impacts of raw materials sourced from for local, regional, or national supply-sheds. For example, to calculate the deforestation footprint of sourcing 100 tons of soy from Argentina, a user could first calculate the average hectares of deforestation per hectare of land use for all soy producing areas in Argentina, and then multiply that value by the land footprint of 100 tons of soy production.

### License

(c) Vizzuality 2023 CC-BY. This data is made available under a Creative Commons Attribution 4.0 International [license](https://creativecommons.org/licenses/by/4.0/).

### Suggested citation

Francis Gassert, Biel Stela, Elena Palao Martinez, and Michael Harfoot. 2023. “Land, Carbon and Biodiversity Data for Supply Chain Impact Calculations.” Preprint. BioRxiv, 2023.11.01.565036. https://doi.org/10.1101/2023.11.01.565036.


Wadhwani AI Pest Management Open Data for Pink Bollworms and American Bollworms

This repository acts as a gateway to the Wadhwani Institute for Artificial Intelligence (Wadhwani AI) pest data set. The data set is primarily intended for training pest recognition models. Such models are used by Wadhwani AI to help cotton farmers make better pest management decisions.
Please view our github repository (https://github.com/WadhwaniAI/pest-management-opendata) which provides detailed documentation about the data and also contains scripts to format the data for consumption by popular modelling frameworks.

computer vision
agriculture
cotton
pink bollworms
american bollworms
wadhwaniai


DEM-Terrain

H3-indexed terrain derivatives computed from the GEDTM-30m global 30m resolution digital elevation model. Includes elevation, slope, aspect, terrain ruggedness index, and topographic position index across ten H3 resolutions with ~10.5 billion cells. Available in Apache Parquet format (v2: BIGINT H3 index; v1: legacy VARCHAR with geometry columns).

remote-sensing digital-elevation-model terrain-derivatives elevation h3-hexagons

Dataset files (20 x .parquet)

Source Cooperative product page

## README.md
# Global DEM Terrain Derivatives

H3-indexed terrain data from [GEDTM-30m](https://stac.openlandmap.org/gedtm-30m/collection.json) in Apache Parquet format. Ten H3 resolutions (1–10), one file per resolution, sorted by `h3_index`. Available in two versions: **v2 (recommended)** with BIGINT `h3_index` and no geometry columns, and **v1 (legacy)** with VARCHAR `h3_index` and [native Parquet 2.11+ GEOMETRY](https://github.com/apache/parquet-format/blob/master/Geospatial.md).

| | |
|---|---|
| **Source** | GEDTM-30m (OpenGeoHub), 30m resolution, global land coverage |
| **Format** | Apache Parquet — v2: BIGINT h3_index, no geometry; v1: VARCHAR h3_index, native GEOMETRY (DuckDB 1.5) |
| **CRS** | EPSG:4326 (WGS 84) |
| **License** | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) by [walkthru.earth](https://walkthru.earth/links) |
| **Code** | [walkthru-earth/dem-terrain](https://github.com/walkthru-earth/dem-terrain) |

## Quick Start

### v2 (recommended)

```sql
-- DuckDB — no spatial extension needed for v2
INSTALL h3 FROM community; LOAD h3;
INSTALL httpfs; LOAD httpfs;
SET s3_region = 'us-west-2';

SELECT h3_index,
       h3_cell_to_lat(h3_index) AS lat,
       h3_cell_to_lng(h3_index) AS lng,
       elev, slope, aspect
FROM read_parquet('s3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v2/h3/h3_res=5/data.parquet')
WHERE h3_index BETWEEN ? AND ?  -- row group statistics prune efficiently
ORDER BY elev DESC
LIMIT 20;
```

For deck.gl H3HexagonLayer (needs hex string):

```sql
SELECT h3_h3_to_string(h3_index) AS h3_hex, elev
FROM read_parquet('s3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v2/h3/h3_res=5/data.parquet')
```

```python
# Python (v2)
import duckdb

con = duckdb.connect()
con.install_extension("h3", repository="community"); con.load_extension("h3")
for ext in ("httpfs",):
    con.install_extension(ext); con.load_extension(ext)
con.sql("SET s3_region = 'us-west-2'")

df = con.sql("""
    SELECT h3_index,
           h3_cell_to_lat(h3_index) AS lat,
           h3_cell_to_lng(h3_index) AS lng,
           elev, slope, aspect, tri, tpi
    FROM read_parquet(
        's3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v2/h3/h3_res=5/data.parquet'
    )
    ORDER BY elev DESC
    LIMIT 100
""").fetchdf()
```

### v1 (legacy — requires `INSTALL spatial`)

```sql
-- DuckDB
INSTALL spatial; LOAD spatial;
INSTALL httpfs;  LOAD httpfs;
SET s3_region = 'us-west-2';

SELECT h3_index, elev, slope, aspect, tri, tpi
FROM read_parquet('s3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v1/h3/h3_res=5/data.parquet')
WHERE lat BETWEEN 35 AND 45
  AND lon BETWEEN -10 AND 5
ORDER BY elev DESC
LIMIT 20;
```

## Files

### v2 (recommended)

```
walkthru-earth/dem-terrain/
  v2/h3/
    h3_res=1/data.parquet       12 KB           223 cells
    h3_res=2/data.parquet       57 KB         1,546 cells
    h3_res=3/data.parquet      373 KB        10,851 cells
    h3_res=4/data.parquet      2.5 MB        76,135 cells
    h3_res=5/data.parquet       17 MB       533,062 cells
    h3_res=6/data.parquet      115 MB     3,730,922 cells
    h3_res=7/data.parquet      783 MB    26,115,785 cells
    h3_res=8/data.parquet      5.3 GB   182,814,924 cells
    h3_res=9/data.parquet     36.1 GB 1,279,700,961 cells
    h3_res=10/data.parquet   244.7 GB 8,957,910,337 cells
```

### v1 (legacy)

```
walkthru-earth/dem-terrain/
  v1/h3/
    h3_res=1/data.parquet  ...  h3_res=10/data.parquet
```

Same cell counts and resolutions as v2.

**Total: 10,450,894,746 cells (~10.5 billion) in ~287 GB.** Compression: ZSTD level 3. Row groups: 1,000,000 rows.

## Schema

### v2 (recommended)

| Column | Type | Unit | Description |
|--------|------|------|-------------|
| `h3_index` | BIGINT | — | H3 cell ID (int64). Sorted for delta encoding compression and range-based queries. |
| `elev` | FLOAT | meters | Elevation above sea level |
| `slope` | FLOAT | degrees | Terrain slope (0 = flat, 90 = cliff) |
| `aspect` | FLOAT | degrees | Slope direction (0/360 = N, 90 = E, 180 = S, 270 = W) |
| `tri` | FLOAT | meters | Terrain Ruggedness Index — mean abs. elevation diff to 8 neighbors |
| `tpi` | FLOAT | meters | Topographic Position Index — elevation minus mean of 8 neighbors (+ridge, -valley) |

`geometry`, `lat`, and `lon` columns are removed in v2 — derive them via the DuckDB h3 extension: `h3_cell_to_lat(h3_index)`, `h3_cell_to_lng(h3_index)`, `h3_cell_to_boundary_wkt(h3_index)`.

### v1 (legacy)

| Column | Type | Unit | Description |
|--------|------|------|-------------|
| `h3_index` | VARCHAR | — | H3 cell ID (hex string) |
| `geometry` | GEOMETRY | — | Cell center point (native Parquet 2.11+ GEOMETRY, EPSG:4326) |
| `lat` | FLOAT | degrees | Cell center latitude |
| `lon` | FLOAT | degrees | Cell center longitude |
| `elev` | FLOAT | meters | Elevation above sea level |
| `slope` | FLOAT | degrees | Terrain slope (0 = flat, 90 = cliff) |
| `aspect` | FLOAT | degrees | Slope direction (0/360 = N, 90 = E, 180 = S, 270 = W) |
| `tri` | FLOAT | meters | Terrain Ruggedness Index — mean abs. elevation diff to 8 neighbors |
| `tpi` | FLOAT | meters | Topographic Position Index — elevation minus mean of 8 neighbors (+ridge, -valley) |

**Sample values** (res 5, Everest region):

| h3_index | elev | slope | aspect | tri | tpi |
|----------|------|-------|--------|-----|-----|
| 853c0317fffffff | 6,879 m | 40.8° | 134° | 327 m | 52 m |
| 853c03bbfffffff | 6,568 m | 23.0° | 221° | 223 m | 31 m |

## How It Works

Terrain derivatives are computed from the GEDTM-30m elevation raster:

| Derivative | Method | Reference |
|------------|--------|-----------|
| **Slope** | `arctan(√(dz/dx² + dz/dy²))` via central differences, pixel sizes converted to meters using latitude-dependent scale | Horn (1981) |
| **Aspect** | `atan2(-dz/dx, dz/dy)`, converted to 0–360° compass bearing | Horn (1981) |
| **TRI** | Mean absolute elevation difference across 3×3 neighborhood | Riley et al. (1999) |
| **TPI** | Center pixel minus mean of 3×3 neighborhood | Weiss (2001) |

Values at H3 cell centers are obtained by **bilinear interpolation** from the raster grid (scipy `RegularGridInterpolator`).

DEM is read at different resolutions depending on H3 level:

| H3 Res | DEM Sampling | Rationale |
|--------|-------------|-----------|
| 1–5 | ~500 m | Cells are 8–418 km; coarse DEM sufficient |
| 6–7 | ~110 m | Cells are 1.2–3.2 km; finer detail needed |
| 8–10 | ~30 m (native) | Cells are 66–461 m; full resolution |

## More Examples

### v2 queries

```sql
-- Range query on sorted BIGINT h3_index — row group min/max statistics prune efficiently
INSTALL h3 FROM community; LOAD h3;
INSTALL httpfs; LOAD httpfs;
SET s3_region = 'us-west-2';

SELECT h3_index,
       h3_cell_to_lat(h3_index) AS lat,
       h3_cell_to_lng(h3_index) AS lng,
       elev, slope
FROM read_parquet('s3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v2/h3/h3_res=7/data.parquet')
WHERE h3_index BETWEEN ? AND ?
ORDER BY elev DESC
LIMIT 100;

-- Query across all resolutions (Hive partitioning)
SELECT h3_res, h3_index, elev
FROM read_parquet(
    's3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v2/h3/h3_res=*/data.parquet',
    hive_partitioning = true
)
WHERE h3_res = 5
ORDER BY elev DESC
LIMIT 20;

-- DuckDB-WASM (browser) — use HTTPS URL
SELECT h3_index,
       h3_cell_to_lat(h3_index) AS lat,
       h3_cell_to_lng(h3_index) AS lng,
       elev, slope
FROM read_parquet(
    'https://data.source.coop/walkthru-earth/dem-terrain/v2/h3/h3_res=5/data.parquet'
)
ORDER BY elev DESC
LIMIT 100;
```

### v1 legacy queries (requires `INSTALL spatial`)

```sql
-- Spatial pushdown using native geometry bbox stats (no lat/lon filter needed)
SELECT h3_index, elev, slope
FROM read_parquet('s3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v1/h3/h3_res=7/data.parquet')
WHERE geometry && ST_MakeEnvelope(86.5, 27.5, 87.5, 28.5);

-- lat/lon filtering (v1 only — these columns exist in v1)
SELECT *
FROM read_parquet(
    's3://us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v1/h3/h3_res=*/data.parquet',
    hive_partitioning = true
)
WHERE h3_res = 5
  AND lat BETWEEN 27 AND 29 AND lon BETWEEN 86 AND 88;
```

## Geometry Format

**v2** drops the `geometry` column entirely. Coordinates are derivable from the BIGINT `h3_index` via the DuckDB h3 extension (`h3_cell_to_lat`, `h3_cell_to_lng`, `h3_cell_to_boundary_wkt`). This eliminates the need for `INSTALL spatial` and reduces file size.

**v1** retains the `geometry` column, written with `GEOPARQUET_VERSION 'BOTH'`, providing dual compatibility:

- **Native Parquet 2.11+ GEOMETRY logical type** — per-row-group bounding box statistics for spatial filter pushdown. Supported by DuckDB 1.5+, Apache Arrow (Rust), Apache Iceberg, GDAL 3.12+.
- **GeoParquet 1.0 `geo` file-level metadata** — backwards compatibility with older tools (QGIS, pyarrow, GeoPandas).

## Source

[GEDTM-30m](https://stac.openlandmap.org/gedtm-30m/collection.json) — Global Ensemble Digital Terrain Model at 30m resolution. Cloud Optimized GeoTIFF, 1,440,010 × 600,010 px, Int32 (decimeters, scale 0.1), EPSG:4326, global land coverage 65°S–85°N.

> Ho, Y., Grohmann, C. H., Lindsay, J., Reuter, H. I., Parente, L., Witjes, M., & Hengl, T. (2025). GEDTM30: global ensemble digital terrain model at 30 m and derived multiscale terrain variables. *PeerJ*, 13, e19673. [doi:10.7717/peerj.19673](https://doi.org/10.7717/peerj.19673)
>
> Dataset: [doi:10.5281/zenodo.14900181](https://doi.org/10.5281/zenodo.14900181)

## License

This dataset is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) by [walkthru.earth](https://walkthru.earth/links). The source [GEDTM-30m](https://doi.org/10.5281/zenodo.14900181) is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) by [OpenGeoHub](https://opengeohub.org/).


Walkthru-Indices

A suite of four interoperable global planetary indices — terrain morphology (elevation, slope, aspect, TRI, TPI), building density and volume, SSP2 population projections (2025–2100), and near-real-time weather forecasts — derived from scientific source datasets and aggregated onto a unified H3 hexagonal grid at multiple resolutions. All indices are stored as Apache Parquet files partitioned by H3 resolution and share a common h3_index key, enabling cross-dataset spatial joins in a single SQL statement. Source data includes GEDTM-30m elevation, Global Building Atlas (2.75 billion building polygons), WorldPop SSP rasters, and NOAA AI-NWP forecasts.

h3 global-coverage building-footprints digital-elevation-model numerical-weather-prediction demographics topography elevation

city_index.parquet

country

[placename] Country name for the address record.

region

[placename] Sub-national region or administrative area name.

city

[placename] City name for the address record.

addresses-index/v1/city_index

region

[placename] Sub-national region name for the city index record.

city

[placename] City name for the city index record.

[latlon] Western bounding longitude of city extent (integer degrees × 10⁶).

[latlon] Eastern bounding longitude of city extent (integer degrees × 10⁶).

[latlon] Southern bounding latitude of city extent (integer degrees × 10⁶).

[latlon] Northern bounding latitude of city extent (integer degrees × 10⁶).

country

[placename] Country name for the city index record.

4 files sampled for this table

weather

[spatial_index] H3 cell identifier (BIGINT int64) encoding the geographic location of each weather record.

5 files sampled for this table

Source Cooperative product page

## README.md
# walkthru-earth / indices

Open planetary indices that turn raw scientific data into a single, H3-hexagonal grid anyone can query with one SQL statement. Every index shares the same cell ID (`h3_index`), so joining terrain + buildings + population + weather is a one-line `JOIN ON h3_index`.

**Why?** Cities shape how people feel — but the data to prove it is scattered across dozens of agencies, formats, and projections. We unify it into one queryable, open layer so researchers, planners, and communities can finally see the full picture. Read more: [walkthru.earth/hormones-cities](https://walkthru.earth/hormones-cities)

```mermaid
flowchart LR
    subgraph Raw["Raw sources"]
        DEM_RAW["GEDTM-30m\n(30 m GeoTIFF)"]
        GBA_RAW["Global Building Atlas\n(2.75 B polygons)"]
        WP_RAW["WorldPop SSP\n(1 km rasters)"]
        NOAA_RAW["NOAA AI-NWP\n(0.25° NetCDF)"]
    end

    subgraph H3["H3 conversion"]
        direction TB
        DT["dem-terrain\nelev · slope · aspect · TRI · TPI"]
        BI["walkthru-building-index\ndensity · height · volume"]
        PI["walkthru-pop-index\n2025 → 2100 projections"]
        WI["walkthru-weather-index\ntemp · wind · precip · pressure"]
    end

    DEM_RAW --> DT
    GBA_RAW --> BI
    WP_RAW --> PI
    NOAA_RAW --> WI
    DT -. topo corrections .-> WI

    subgraph S3["s3://…/walkthru-earth/"]
        T[dem-terrain/v2/h3/]
        B[indices/building/v2/h3/]
        P[indices/population/v2/]
        W[indices/weather/]
    end

    DT --> T
    BI --> B
    PI --> P
    WI --> W

    T -. JOIN h3_index .-> Q((Query))
    B -. JOIN .-> Q
    P -. JOIN .-> Q
    W -. JOIN .-> Q
```

## Indices

| Index | What | H3 Res | Update | Size |
|-------|------|--------|--------|------|
| **[dem-terrain](https://source.coop/walkthru-earth/dem-terrain)** | Elevation, slope, aspect, TRI, TPI from GEDTM-30m | 1–10 | One-time | ~287 GB, 10.5 B cells |
| **[walkthru-building-index](https://source.coop/walkthru-earth/indices)** | Density, height, footprint, volume from 2.75 B buildings | 3–8 | One-time | ~2.6 GB, 53 M cells |
| **[walkthru-pop-index](https://source.coop/walkthru-earth/indices)** | SSP2 projections 2025–2100 (16 timesteps) from WorldPop | 1–8 | One-time | ~4.5 GB, 44 M cells |
| **[walkthru-weather-index](https://source.coop/walkthru-earth/indices)** | Temperature, wind, precip, pressure + topo corrections | 0–5 | Every 12 h | ~3.6 GB / forecast |

All datasets: [Apache Parquet](https://parquet.apache.org/), ZSTD compression, sorted by `h3_index`.
- **v2 (recommended):** `h3_index` is BIGINT (int64) — no `geometry`, `lat`, `lon`, or `area_km2` columns (all derivable from the H3 index via DuckDB's h3 extension). Sorted by int64 for better compression and fast range-based spatial queries.
- **v1 (legacy):** `h3_index` is VARCHAR (hex string) with native Parquet 2.11+ GEOMETRY and `lat`/`lon`/`area_km2` columns.

## Quick start

```sql
-- DuckDB: join buildings + population in one query (v2 — BIGINT h3_index, no geometry)
INSTALL httpfs;  LOAD httpfs;
INSTALL h3 FROM community; LOAD h3;
SET s3_region = 'us-west-2';

SELECT b.h3_index,
       h3_h3_to_string(b.h3_index) AS h3_hex,          -- hex string for deck.gl
       h3_cell_to_lat(b.h3_index)  AS lat,              -- derive lat on the fly
       h3_cell_to_lng(b.h3_index)  AS lon,              -- derive lon on the fly
       b.building_count, b.building_density,
       p.pop_2025, p.pop_2050,
       (b.building_count::FLOAT / NULLIF(p.pop_2025, 0))::DECIMAL(6,3) AS bldg_per_person
FROM read_parquet('s3://us-west-2.opendata.source.coop/walkthru-earth/indices/building/v2/h3/h3_res=5/data.parquet') b
JOIN read_parquet('s3://us-west-2.opendata.source.coop/walkthru-earth/indices/population/v2/scenario=SSP2/h3_res=5/data.parquet') p
  ON b.h3_index = p.h3_index                            -- BIGINT join — fast
WHERE p.pop_2025 > 10000
ORDER BY bldg_per_person DESC
LIMIT 10;
```

## S3 layout

```
s3://us-west-2.opendata.source.coop/walkthru-earth/
├── dem-terrain/
│   ├── v1/h3/h3_res={1-10}/data.parquet          ← legacy (VARCHAR h3, has geometry/lat/lon)
│   └── v2/h3/h3_res={1-10}/data.parquet          ← recommended (BIGINT h3, lean schema)
└── indices/
    ├── building/
    │   ├── v1/h3/h3_res={3-8}/data.parquet        ← legacy
    │   ├── v2/h3/h3_res={3-8}/data.parquet        ← recommended
    │   └── globalbuildingatlas/zoomlevel={2-14}/data*.parquet
    ├── population/
    │   ├── v1/scenario=SSP2/h3_res={1-8}/data.parquet  ← legacy
    │   └── v2/scenario=SSP2/h3_res={1-8}/data.parquet  ← recommended
    └── weather/
        └── model=GraphCast_GFS/date=YYYY-MM-DD/hour={0,12}/h3_res={0-5}/data.parquet
```

## License

[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) by [walkthru.earth](https://walkthru.earth/links)

extent.spatial.bbox: omitted; spatial reference is categorical via [weather.h3_index (spatial_index), addresses-index/v1.country (placename), addresses-index/v1.region (placename), addresses-index/v1.city (placename), addresses-index/v1/city_index.country (placename), addresses-index/v1/city_index.region (placename), addresses-index/v1/city_index.city (placename), addresses-index/v1/city_index.bbox_min_lon_e6 (latlon), addresses-index/v1/city_index.bbox_max_lon_e6 (latlon), addresses-index/v1/city_index.bbox_min_lat_e6 (latlon), addresses-index/v1/city_index.bbox_max_lat_e6 (latlon)]. Add bbox once the placename → bbox gazetteer covers it.


OpenSensor.Space

Continuous time-series environmental and weather readings from an IoT sensor network in the Cairo/Nile Delta region of Egypt. Data is collected via Raspberry Pi devices with Enviro+ sensors, capturing temperature, pressure, humidity, gas concentrations (oxidised, reducing, NH3), ambient light, and particulate matter (PM1, PM2.5, PM10) at multiple particle size fractions, partitioned by station and date. A companion asset provides geospatial suitability analysis for air quality monitoring station placement.

air-quality environmental-monitoring time-series weather particulate-matter sensor-network egypt

enviroplus

40 files sampled for this table

suitability_analysis_of_aq.parquet

geom

[geometry] OGC CRS84 geometry for candidate air quality monitoring station suitability locations

Source Cooperative product page

extent.spatial.bbox: omitted; spatial reference is categorical via [share.geom (geometry)]. Add bbox once the placename → bbox gazetteer covers it.


Fields of the World

High-resolution Sentinel-2 mosaics and AI-derived field-boundary probability maps for Japan, Mexico, Rwanda, South Africa, and Switzerland. This repository contains cloud-optimized Zarr datasets produced using the WherobotsAI platform during the FTW Phase 2 Model Bakeoff. Includes bi-temporal mosaics (Planting and Harvest seasons) and raw softmax model outputs for agricultural field segmentation.


Global Human Modification Dataset (1990–2017)

This dataset provides a global, high-resolution (300-meter) view of how human activities have modified terrestrial landscapes from 1995 to 2017, covering all land areas except Antarctica. It offers a continuous, quantitative estimate of human modification (H) ranging from 0 (no modification) to 1 (complete modification).


US Structures from Oak Ridge National Laboratory

A comprehensive geospatial database of over 125 million building structures across the United States and its territories, covering every structure larger than 450 square feet. Produced by Oak Ridge National Laboratory (ORNL) using AI, high-performance computing, and over 1.1 petabytes of satellite and aerial imagery. Each record includes building footprint geometry, occupancy classification, property address, physical dimensions (height, floor area), and elevation data.

building-footprints remote-sensing land-use geoparquet pmtiles machine-learning aerial-imagery

geoparquet

geometry

[geometry] Building footprint polygon geometry in EPSG:4326

[placename] Property city name

[admin_code] Property state abbreviation code

[zipcode] Property ZIP code

FIPS

[admin_code] FIPS county or state code

USNG

[spatial_index] US National Grid reference for the building location

LONGITUDE

[latlon] Centroid longitude of building structure

LATITUDE

[latlon] Centroid latitude of building structure

bbox

STRUCT(xmin DOUBLE, ymin DOUBLE, xmax DOUBLE, ymax DOUBLE)

geohash

[spatial_index] Geohash spatial index for the building location

5 files sampled for this table

iceberg/data

[placename] Property city name

[zipcode] Property ZIP code

FIPS

[admin_code] FIPS county or state code

geom

[geometry] Building footprint geometry in OGC:CRS84

5 files sampled for this table

Source Cooperative product page

## README.md
# USA Structures Dataset

The **USA Structures Dataset** provides a comprehensive geospatial database of over 125 million building structures across the U.S. and its territories, capturing every structure larger than 450 square feet. Developed by Oak Ridge National Laboratory (ORNL) through the integration of advanced geospatial science, high-performance computing, and AI, this dataset processes over 1.1 petabytes of imagery. It serves as a valuable resource across various fields, supporting tasks from urban planning to emergency response with detailed location-based data. Its precise, accessible structure facilitates cross-agency collaboration, making it an essential tool for federal, local, state, tribal, and territorial organizations. The dataset accelerates tasks like damage assessments and recovery operations, aiding communities in recovering from disasters efficiently.

Special thank you to [Mark Litwintschik](https://www.linkedin.com/feed/update/urn:li:activity:7132210177289650176/?updateEntityUrn=urn%3Ali%3Afs_feedUpdate%3A%28V2%2Curn%3Ali%3Aactivity%3A7132210177289650176%29) and his [original post](https://tech.marksblogg.com/ornl-fema-buildings.html) that outlined the initial ETL process.

---

## Quickstart Guide

Access to this dataset is available in **GeoParquet**, **Wherobots Havasu (Iceberg)**, and **PMTiles**.

### GeoParquet

You can access the GeoParquet data directly as partitioned GeoParquet files. These are partitioned by the primary occupancy which includes:

- Education
- Residential
- Government
- Utility and Misc
- Industrial
- Agriculture
- Unclassified
- Assembly
- Commercial

These files are located in `/geoparquet`.

**Example code**

```python
df = sedona.read.format("geoparquet").load(YOUR_S3_SOURCE_COOP_URI + 'geoparquet/OCC_CLS=Residential/*.parquet)
df.printSchema()
```

### Wherobots Havasu

Havasu is a spatial table format built on top of Apache Iceberg. WherobotsDB can save its in-memory geometry and raster tables to the Havasu table format and store them on AWS S3, or any other cloud file systems or object stores.

Havasu supports ACID transactions, schema evolution, and time travel on spatial data including geometries and rasters. Havasu stores spatial data in Apache Parquet format on cloud storage, which is very cost effective and highly decoupled with computation.

To use these files first copy the files to an S3 bucket of your choosing. You can get access to a bucket by signing up for a Wherobots account [here](https://cloud.wherobots.com/auth/register).

Once you have done that you can add a new spatial catalog using [these instructions](https://docs.wherobots.com/latest/develop/storage-management/s3-storage-integration/?h=catalog#spatial-catalog).

If you are using your own S3 bucket please refer to [these instructions](https://docs.wherobots.com/latest/develop/storage-management/s3-storage-integration/).

## PMTiles

You can use the PMTiles URL in any application or tool that supports PMTiles such as Felt, leafmap, MapLibre, and the [PM Tiles Viewer](https://pmtiles.io/)

extent.spatial.bbox: omitted; spatial reference is categorical via [geoparquet.geometry (geometry), geoparquet.LATITUDE (latlon), geoparquet.LONGITUDE (latlon), geoparquet.PROP_CITY (placename), geoparquet.PROP_ST (admin_code), geoparquet.PROP_ZIP (zipcode), geoparquet.FIPS (admin_code), geoparquet.geohash (spatial_index), geoparquet.USNG (spatial_index), iceberg/data.geom (geometry), iceberg/data.LATITUDE (latlon), iceberg/data.LONGITUDE (latlon), iceberg/data.PROP_CITY (placename), iceberg/data.PROP_ST (admin_code), iceberg/data.PROP_ZIP (zipcode), iceberg/data.FIPS (admin_code), iceberg/data.USNG (spatial_index)]. Add bbox once the placename → bbox gazetteer covers it.


The Wildland Almanac - California

The Wildland Almanac - California: a 30 m Landsat-derived time series of California wildland ecosystem properties across forests, shrublands, and grasslands; annual water years 1985–2025 (41 years). 18 properties spanning six themes - vegetation cover, hydrology, fire hazard, carbon, drought-driven dieoff risk, and disturbance severity. The Fire_LCP landscape is delivered as eight single-band COGs (fuel model, canopy cover, canopy height, canopy base height, canopy bulk density, plus static elevation, slope, and aspect). Created using a unified methodology that emphasizes temporal and cross-property consistency, to track change, explore tradeoffs, and support decision-making and scientific discovery. Cloud-Optimized GeoTIFFs with a STAC catalog. CC BY 4.0.


The Wildland Almanac - CONUS

The Wildland Almanac - CONUS: a 30 m Landsat-derived time series of conterminous-U.S. wildland ecosystem properties across forests, shrublands, and grasslands; decadal snapshots for water years 1990, 2000, 2010, 2020, and 2024. 17 properties spanning five themes - vegetation cover, hydrology, fire hazard, carbon, and past disturbance severity. The Fire_LCP landscape is delivered as eight single-band COGs (fuel model, canopy cover, canopy height, canopy base height, canopy bulk density, plus static elevation, slope, and aspect). Disturbance is cumulative for the period between snapshots. Created using a unified methodology that emphasizes temporal and cross-property consistency, to track change, explore tradeoffs, and support decision-making and scientific discovery. Cloud-Optimized GeoTIFFs with a STAC catalog. CC BY 4.0.


The Wildland Almanac - Treatment Scenarios: a 30 m land management "what-if" companion to Wildland Almanac – Western CONUS for forests, shrublands, and grasslands. Considers 24 alternative forest and fuel treatment types, and provides raster predictions of the conditions expected in the first year after treatment. Conditions include evapotranspiration, gross primary production, aboveground biomass, drought-driven dieoff risk, and fire hazard (flame length and rate of spread). Predictions are anchored to the observed 2024 baseline conditions in the Wildland Almanac. Created using a unified methodology that emphasizes past-to-future and cross-property consistency, with the goal of creating an integrated framework that supports both planning and tracking progress. Designed and built in collaboration with https://www.blueforest.org/. Cloud-Optimized GeoTIFFs with a STAC catalog. CC BY 4.0.


Global Rangelands Raster

A global raster map (version 1) delineating the extent of rangelands worldwide, compiled and created by Dr. Mika Tosca at Woodwell Climate Research Center. The dataset consists of 12 GeoTIFF files covering the full global extent at approximately 486 MB total.

land-use land-cover rangelands grasslands global-coverage

Dataset files (12 x .tif)

Source Cooperative product page

license: AI returned 'unknown' (README does not state a license). Verify with publisher before flipping to _state: reviewed.

providers: role [producer] for 'Woodwell Climate Research Center' is an AI guess — Assumed producer based on API description attributing creation to Dr. Mika Tosca at this institution; licensor role not stated. Verify before flipping to _state: reviewed.


Secondary Forest Brazil — YbYrá-BR v8.1 (from MapBiomas Collection 10.1, 1986–2024)

Annual maps of secondary forest dynamics in Brazil (1986–2024) derived from MapBiomas Collection 10.1 land-use and land-cover data processed in Google Earth Engine at 30 m resolution. Four products are provided: age (years of continuous secondary forest since last disturbance), extent (binary presence/absence), increment (new forest pixels per year), and loss (cleared pixels per year), covering Forest Formation and Flooded Forest classes.

remote-sensing land-use land-cover secondary-forest forest-dynamics deforestation conservation

Dataset files (155 x .tif)

Source Cooperative product page

## README.md
# Brazilian Secondary Forest Dynamics (1986–2024)

Annual maps of increment, extent, age, and loss of secondary forests in Brazil, derived from MapBiomas Collection 10.1 land-use and land-cover data using Google Earth Engine. The **Forest Formation** and **Flooded Forest** classes are included.

## Products

| Product | Description | Period |
|:---|:---|:---:|
| **Age** | Years of continuous secondary forest since last disturbance | 1986–2024 |
| **Extent** | Binary presence/absence of secondary forest | 1986–2024 |
| **Increment** | New secondary forest pixels per year | 1986–2024 |
| **Loss** | Secondary forest pixels cleared per year | 1987–2024 |

## Technical Specifications

| Parameter | Value |
|:---|:---:|
| Spatial resolution | 30 m |
| CRS | EPSG:4326 (WGS84) |
| Format | Cloud-Optimized GeoTIFF (COG) |
| Input data | MapBiomas Collection 10.1 |
| Forest classes | Forest Formation, Flooded Forest |
| Version | v8.1 |

## GEE Assets

Multi-band assets (one band per year, named `classification_YYYY`) are publicly available:

```javascript
var age       = ee.Image('users/ybyrabr/public/secondary_forest_age_collection10_1_v8_1');
var extent    = ee.Image('users/ybyrabr/public/secondary_forest_extent_collection10_1_v8_1');
var increment = ee.Image('users/ybyrabr/public/secondary_forest_increment_collection10_1_v8_1');
var loss      = ee.Image('users/ybyrabr/public/secondary_forest_loss_collection10_1_v8_1');
```

## Code & STAC

- **GitHub:** https://github.com/celsohlsj/gee_brazil_sv
- **STAC Catalog:** https://celsohlsj.github.io/ybyrastac/

## Citation

```bibtex
@article{silvajunior2020,
  author  = {Silva-Junior, Celso H. L. and Heinrich, Viola H. A. and Freire, Ana T. G.
             and Broggio, Igor S. and Rosan, Thais M. and Doblas, Juan and Anderson,
             Liana O. and Rousseau, Guillaume X. and Shimabukuro, Yosio E. and
             Silva, Carlos A. and House, Joanna I. and Aragão, Luiz E. O. C.},
  title   = {Benchmark maps of 33 years of secondary forest age for {Brazil}},
  journal = {Scientific Data},
  year    = {2020},
  doi     = {10.1038/s41597-020-00600-4}
}
```

## Funding & License

Produced by the **YBYRÁ-BR Project**, funded by CNPq (Process 401741/2023-0).
Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).

providers: role [producer] for 'MapBiomas' is an AI guess — Produced the Collection 10.1 LULC input data; role inferred from README attribution as upstream data source, not explicitly stated.. Verify before flipping to _state: reviewed.


Cloud-Native Geocoding

This project showcases the integration of DuckDB with Lowcoder to perform efficient and serverless geocoding within a cloud-native geospatial formats like geoparquet. (the current data is for Egypt)

geoparquet geocoding cloud-native duckdb lowcoder


European Ground Motion Cloud Native Data

Ground deformation measurements for Europe derived from synthetic aperture radar data, provided by the Copernicus European Ground Motion Service. Data has been converted to cloud-native Parquet Geo and PMTiles formats for scalable analysis and web mapping.

remote-sensing ground-motion deformation-monitoring synthetic-aperture-radar

Dataset files (48 x .parquet)

Source Cooperative product page

## README.md
# European Ground Motion Service Data Conversion

This repository contains data converted from the original Copernicus European Ground Motion Service datasets. The source data was provided as zipped CSV files and has been transformed into cloud native geospatial formats.

## Data Sources

- **Original Data**: [Copernicus European Ground Motion Service](https://land.copernicus.eu/en/products/european-ground-motion-service)
- **Formats Provided**:
  - **Parquet Geo** – Optimized for scalable geospatial analytics.
  - **PMTiles** – Designed for efficient, cloud-native tile serving.

## Conversion Process

1. **Extraction**: Unzipping and parsing CSV files.
2. **Transformation**: Converting CSV data into geospatial datasets.
3. **Output**: Generating Parquet Geo files for analysis and PMTiles(with only pid and geom and filename) for mapping.

## Usage

- **Parquet Geo**: Use with cloud-based analytics platforms for fast querying.
Example using DuckDB:
```sql
SELECT * FROM read_parquet('https://data.source.coop/youssef-harby/egms-copernicus/L2a/parquet/CY/EGMS_L2a_087_0205_IW2_VV_2019_2023_1.parquet') LIMIT 10;
```
- **PMTiles**: Serve directly to web maps or GIS applications for interactive visualization like [here](https://pmtiles.io/?url=https%3A%2F%2Fdata.source.coop%2Fyoussef-harby%2Fegms-copernicus%2FL2a%2Fpmtiles%2Fcy.pmtiles#map=9.36/34.9833/33.293).

## License & Acknowledgements

Please refer to the original data license provided by Copernicus for usage terms. This conversion project builds on the high-quality datasets provided by the European Ground Motion Service.


EXIOBASE 3

EXIOBASE 3 is a global, environmentally extended multi-regional input-output (EE-MRIO) database covering 44 countries and five rest-of-world regions across 200 sectors from 1995 to 2022 (with nowcasts through 2022). It quantifies economic transactions between regions and sectors alongside environmental pressures including greenhouse gas emissions, energy use, land, water, and material flows. Data are stored in partitioned Apache Parquet files organized by year, format (ixi/pxp), and matrix type.

input-output-tables environmental-footprint emissions trade economic-sectors multi-regional

f_impacts_units_1995_ixi.parquet

f_impacts_units_1995_pxp.parquet

f_impacts_units_1996_ixi.parquet

f_impacts_units_1996_pxp.parquet

f_impacts_units_1997_ixi.parquet

data.parquet

[admin_code] Country/region code for the supply-side region in Z and Y matrices (e.g. 'CN', 'US')

[admin_code] Country/region code for the demand-side region in Z and Y matrices

Source Cooperative product page

## README.md
# EXIOBASE 3: Global Input-Output Database

EXIOBASE 3 is a global, multi-regional input-output database, offering a detailed view of the world economy and its environmental impacts. It covers 49 regions and 200 sectors from 1995 to 2022. This dataset is ideal for analyzing trade, economic structures, and environmental footprints.

## Data Structure

The data is stored in Parquet format and partitioned by `year`, `format` (ixi/pxp), and `matrix` type (Z, Y, F_satellite, F_impacts) to allow for efficient querying.

```
s3://us-west-2.opendata.source.coop/youssef-harby/exiobase-3/
└── 4588235/
    ├── metadata/
    │   ├── f_impacts_units_1995_ixi.parquet
    │   ├── ... (112 files)
    └── parquet/
        ├── year={1995,1996,...,2022}/
        │   ├── format={ixi,pxp}/
        │   │   ├── matrix={F_impacts,F_satellite,Y,Z}/
        │   │   │   └── data.parquet
```

## Quickstart with DuckDB

The following examples demonstrate how to effectively query the EXIOBASE 3 dataset using DuckDB.

### A Note on Performance

For complex queries, such as time-series analysis or combining large matrices, direct queries may time out. A more robust approach is to break down the query into smaller parts and combine the results in your client application (e.g., Python with Pandas, R, etc.). The examples below follow this best practice.

### Top 10 Exporters to the US (2022)

To calculate the total exports to the United States, you should query the inter-industry (`Z`) and final demand (`Y`) matrices separately and then sum the results.

**Step 1: Get Exports from Inter-Industry Transactions**
```sql
SELECT origin_region, SUM(value) AS total_z
FROM 's3://us-west-2.opendata.source.coop/youssef-harby/exiobase-3/4588235/parquet/year=2022/format=ixi/matrix=Z/data.parquet'
WHERE dest_region = 'US' AND origin_region <> 'US' AND origin_region NOT LIKE 'W%'
GROUP BY origin_region
ORDER BY total_z DESC
LIMIT 20;
```

**Step 2: Get Exports from Final Demand**
```sql
SELECT origin_region, SUM(value) AS total_y
FROM 's3://us-west-2.opendata.source.coop/youssef-harby/exiobase-3/4588235/parquet/year=2022/format=ixi/matrix=Y/data.parquet'
WHERE dest_region = 'US' AND origin_region <> 'US' AND origin_region NOT LIKE 'W%'
GROUP BY origin_region
ORDER BY total_y DESC
LIMIT 20;
```
*Combine the `total_z` and `total_y` results for each `origin_region` in your client to get the final ranking.*

### China-US Trade (1995-2022)

Similarly, for time-series analysis, query each matrix across all years and then join the results.

**Step 1: Get Time-Series from Inter-Industry Transactions**
```sql
SELECT year, SUM(value) AS z_value
FROM read_parquet('s3://us-west-2.opendata.source.coop/youssef-harby/exiobase-3/4588235/parquet/year=*/format=ixi/matrix=Z/data.parquet', hive_partitioning=1)
WHERE origin_region = 'CN' AND dest_region = 'US' AND year BETWEEN 1995 AND 2022
GROUP BY year
ORDER BY year;
```

**Step 2: Get Time-Series from Final Demand**
```sql
SELECT year, SUM(value) AS y_value
FROM read_parquet('s3://us-west-2.opendata.source.coop/youssef-harby/exiobase-3/4588235/parquet/year=*/format=ixi/matrix=Y/data.parquet', hive_partitioning=1)
WHERE origin_region = 'CN' AND dest_region = 'US' AND year BETWEEN 1995 AND 2022
GROUP BY year
ORDER BY year;
```
*Join the results on the `year` column and sum `z_value` and `y_value` to get the total annual exports.*

### Top 20 CO2 Emitters (2022)

This query identifies the top 20 regions by total CO2 emissions, summing all CO2-related stressors from the `F_satellite` matrix.

```sql
SELECT
    region,
    SUM(value) AS total_co2,
    FIRST(unit) AS unit
FROM 's3://us-west-2.opendata.source.coop/youssef-harby/exiobase-3/4588235/parquet/year=2022/format=ixi/matrix=F_satellite/data.parquet'
WHERE stressor LIKE '%CO2%'
GROUP BY region
ORDER BY total_co2 DESC
LIMIT 20;
```

providers: role [producer, licensor] for 'EXIOBASE Consortium' is an AI guess — Original authors of the EXIOBASE database; licensor assumed from consortium ownership of the underlying data. Verify before flipping to _state: reviewed.

providers: role [processor] for 'Youssef Harby' is an AI guess — Converted EXIOBASE source data to Apache Parquet and published to Source Cooperative; inferred from S3 account path and API description. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [data.origin_region (admin_code), data.dest_region (admin_code), data.region (admin_code)]. Add bbox once a placename → bbox gazetteer is available.


Overture Maps STAC

This is a demo of the [Overture Maps Data](https://overturemaps.org/download/) STAC based on https://beta.source.coop/cholmes/overture

overture maps
geoparquet
stac
catalog
metadata


Weather Station Real-Time in Parquet

High-resolution environmental monitoring data collected by a Raspberry Pi with Enviro+ sensor and streamed directly to cloud storage in Parquet format. Each record captures timestamped atmospheric measurements including temperature, pressure, humidity, air quality gases (oxidised, reducing, NH3), light intensity, and particulate matter (PM1, PM2.5, PM10) across one or more named stations. The dataset is organized by station and date, enabling real-time and historical analysis of local atmospheric conditions.

weather air-quality particulate-matter environmental-monitoring real-time climate-change

2025_03_19.parquet

2025_03_20.parquet

2025_03_21.parquet

2025_03_22.parquet

2025_03_23.parquet

data_2355.parquet

latitude

[latlon] Station latitude in decimal degrees

longitude

[latlon] Station longitude in decimal degrees

data_0214.parquet

data_0219.parquet

data_0220.parquet

Source Cooperative product page

providers: role [producer, licensor] for 'OpenSensor Space (opensensor.space)' is an AI guess — Inferred as producer and licensor from the project URL in the API description; no explicit statement in README.. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [data_2355.latitude (latlon), data_2355.longitude (latlon)]. Add bbox once a placename → bbox gazetteer is available.


Tanzania Crop Survey

This dataset was collected by YouthMappers students in three regions of Tanzania: Arusha, Dodoma and Mwanza. It includes estimated field centroid coordinates, primary crop, secondary crop and estimated field size. The data was collected in a 2023 pilot study supported by USDA’s Foreign Agricultural Service and the USAID/GeoCenter to see if university students could conduct field visits and collect ground-truth data for training machine or deep learning models applied to remotely sensed imagery.

youthmappers tanzania ground-truth crops


Example GeoZarr Stores

GeoZarr stores used to test interoperability of GeoZarr data.


LandCoverNet - Zarr

LandCoverNet is a global annual land cover classification training dataset with labels for the multi-spectral satellite imagery from Sentinel-1, Sentinel-2, and Landsat-8 missions in 2018.
This dataset is a Zarr-ified version of LandCoverNet, utilising the V3 Zarr standard.

land cover
segmentation
zarr
sentinel-1
sentinel-2
landsat 8


NYC Citi Bike

New York City Citibike bike-sharing trip data converted to geoparquet format, partitioned by year and month. The dataset includes station locations, pickup times, ride durations, and ridership patterns. Original data inconsistencies have been standardized into two distinct schemas.

bike-sharing urban-mobility trip-patterns station-locations

Dataset files (156 x .parquet)

Source Cooperative product page

## README.md
**Description**:

This dataset is a copy of the New York City Citibike data, which can be found here: https://citibikenyc.com/system-data, made available in the cloud-native geoparquet format partitioned by year and month.

The original data scehma exhibits inconsistencies across source files,particularly in data type definitions.

This dataset has been standardized and divided into twodistinct schemas ('new' and 'old') to best reflect the official Citibike data schema variations.

Specific data transformation steps are documented in my Medium blog post: https://medium.com/@calvinluozhengpei/  from-wall-street-to-central-park-decoding-nycs-citi-bike-patterns-part-1-3155569278cf

Feel free to read and provide feedback on how to make the process more efficient!

**Example Usage**:

Load data with Duckdb in Python:

```
con = duckdb.connect(database = ":memory:")
con.sql("""
SELECT * FROM read_parquet('s3://us-west-2.opendata.source.coop/zluo43/citibike/old_schema_combined_with_geom.parquet/**/*.parquet')
""")

```

Basic EDA in Python:

```
df=con.sql("""
SELECT * FROM read_parquet('s3://us-west-2.opendata.source.coop/zluo43/citibike/old_schema_combined_with_geom.parquet/**/*.parquet')
""").df()
df.info()

```