ForestSignal California (1985–2025)

ForestSignal-CA is a 30-meter resolution, Landsat-derived data cube covering California's forest, shrubland, and grassland ecosystems across 19 thematic layers for water years 1985–2025. Layers span vegetation structure, hydrology, fire hazard, carbon stocks, drought-driven tree die-off vulnerability, and disturbance severity. Built by university scientists using a unified methodology designed for temporal and cross-property consistency to support ecological research and land management.

remote-sensing forest land-cover fire-hazard carbon landsat hydrology biomass

Dataset files (773 x .tif)

Source Cooperative product page

## README.md
# ForestSignal — California

This repository contains the first full upload of the ForestSignal-CA data cube — 19 layers × 41 years (water years 1985–2025). Documentation, STAC catalog, integration demos, and the broader release infrastructure are still being finalized over the coming weeks. A CONUS release will follow. Layout and metadata conventions may change before the formal release; contact mgoulden@uci.edu.

## About ForestSignal

ForestSignal develops 40-year+ data cubes of forest, shrubland, and grassland properties, including vegetation structure, fuel loads, fire hazard, biomass, hydrology, and disturbance history. The data are based on Landsat and use a unified approach that emphasizes temporal and cross-property consistency, with a goal of tracking change and exploring tradeoffs. The data are built and openly shared by university scientists to support decision-making and scientific discovery.

## What's in this repository

19 layers covering water years 1985–2025 (October–September), 30 m resolution, EPSG:5070 (NAD83 / Conus Albers). All files are Cloud-Optimized GeoTIFFs (COGs).

**Vegetation cover and structure**
- `Veg_TreeFrac/` — fractional tree canopy cover
- `Veg_ShrubFrac/` — fractional shrub cover
- `Veg_HerbFrac/` — fractional herbaceous cover
- `Veg_BareFrac/` — fractional bare/non-vegetated cover
- `Veg_CanopyHt/` — tree canopy height

**Hydrology**
- `WaterFlux_AETmax/` — potential evapotranspiration
- `WaterFlux_AETrealized/` — actual evapotranspiration
- `WaterFlux_Soilmoisture/` — soil moisture
- `WaterFlux_Runoff/` — runoff
- `WaterFlux_PminusETmax_SPI0/` — climatological water deficit at SPI-48 = 0

**Fire hazard**
- `Fire_LCP/` — 8-band landscape file (FARSITE v.4 format) for fire-behavior modeling
- `Fire_FlamMap_FL/` — FlamMap-predicted flame length
- `Fire_FlamMap_ROS/` — FlamMap-predicted rate of spread

**Carbon sequestration**
- `Carbon_AGB/` — above-ground biomass
- `Carbon_GPP/` — gross primary production

**Forest dieoff risk**
- `Vulner_TreeDieoff_SPI-2/` — drought-driven tree dieoff vulnerability under SPI-48 = −2

**Disturbance severity** (water years 1986–2024; first and last years cannot be computed)
- `Disturbance_TreeFrac/` — annual tree-canopy loss
- `Disturbance_ShrubFrac/` — annual shrub-canopy loss
- `Disturbance_AGB/` — annual above-ground biomass loss

## Documentation

`ForestSignal_CA_Documentation.pdf` in this repo contains complete layer descriptions, unit conventions, methodology summaries, projection details, and use limitations. **Read it before working with the data quantitatively** — several layers (Fire_LCP bands, Disturbance layers, Veg fractions) use specific scaling conventions that affect any numeric analysis.

## How to access the data

All files are Cloud-Optimized GeoTIFFs in EPSG:5070, 30 m. Three ways to use them, in increasing order of sophistication.

### Streaming (most common — no download needed)

The COG format means rasters can be read directly from cloud storage over HTTP, transferring only the bytes needed for the current view. The URI form differs between ArcGIS Pro and open-source GIS tools.

**For ArcGIS Pro** — use the S3 URI, for example:
```
s3://us-west-2.opendata.source.coop/forestsignal/forestsignal-ca/Carbon_AGB/ForestSignal_CAwide_Carbon_AGB_2025_V260426.tif
```
In ArcGIS Pro: Add Data → From Path → paste the URI.

**For QGIS, GDAL, rasterio, terra** — use the `/vsicurl/` URL, for example:
```
/vsicurl/https://data.source.coop/forestsignal/forestsignal-ca/Carbon_AGB/ForestSignal_CAwide_Carbon_AGB_2025_V260426.tif
```
In QGIS: Layer → Add Layer → Add Raster Layer → paste the URI as the Source.

Both URI forms point at the same underlying file. Source Cooperative's file detail pages show both forms — "Cloud URI" is the S3 form (for ArcGIS), "Source URL" is the HTTPS form (prepend `/vsicurl/` for GDAL-based tools). Substitute the layer name and year for any other file.

### Direct download via HTTPS

Every file has a public URL you can fetch with any HTTP client. No credentials needed.

```
https://data.source.coop/forestsignal/forestsignal-ca/Carbon_AGB/ForestSignal_CAwide_Carbon_AGB_2025_V260426.tif
```

Use `curl`, `wget`, browser save, `Invoke-WebRequest` on Windows PowerShell.

### Bulk download via AWS CLI

For downloading multiple files or whole layer directories, the AWS CLI is fastest. No credentials or AWS account needed — use the `--no-sign-request` flag:

```
# Download a single file
aws s3 cp s3://us-west-2.opendata.source.coop/forestsignal/forestsignal-ca/Carbon_AGB/ForestSignal_CAwide_Carbon_AGB_2025_V260426.tif . --no-sign-request

# Sync an entire layer directory (41 files)
aws s3 sync s3://us-west-2.opendata.source.coop/forestsignal/forestsignal-ca/Carbon_AGB/ ./Carbon_AGB/ --no-sign-request

# Sync everything
aws s3 sync s3://us-west-2.opendata.source.coop/forestsignal/forestsignal-ca/ ./forestsignal-ca/ --no-sign-request
```

AWS CLI is available for Windows, Mac, and Linux at [aws.amazon.com/cli](https://aws.amazon.com/cli/).

## File naming

Files follow the convention `ForestSignal_CAwide_{LayerName}_{YYYY}_V{YYMMDD}.tif`, where `{YYYY}` is the four-digit water year and `V{YYMMDD}` is the processing-batch date. The version stamp is provisional and may move to STAC metadata in the final release.

## License

These data are shared under the [Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/). Use of these data implies consent to the license terms and to the University of California disclaimer reproduced in §6 of the documentation PDF and summarized below. The license restricts use for commercial purposes — see [creativecommons.org/faq/](https://creativecommons.org/faq/) for a discussion of commercial use.

The data are offered with no promise of technical support and with the hope that users will provide feedback to help refine and improve them.

## Legal and terms of access

The materials are offered "as is"; the user assumes any and all risks of any kind or amount in using these materials. Access or use of these data implies consent to the disclaimer in the dataset documentation (§6 of `ForestSignal_CA_Documentation.pdf`), which is the binding statement.

## Citation

> Goulden, M.L. (2026). *ForestSignal CA: 41 years of reconciled forest dynamics for California, water years 1985–2025* (Version V260426). Source Cooperative. [DOI to be assigned]

## Contact and feedback

mgoulden@uci.edu — notes on errors and use cases are especially valuable. Feedback will materially shape the final release.

---

*Preliminary release V260426 (April 2026 build). University of California, Irvine.*

extent.temporal: None (probe sample) widened to ['1985-01-01T00:00:00Z', '2025-01-01T00:00:00Z'] using README-stated coverage ['1985', '2025'].


FTW Field Boundaries


financial-times


An MGRS-based 4km grid for the FTW Benchmark Dataset

A global grid dataset structured on MGRS cells at 4km resolution, derived from the FTW Benchmark Dataset. Comprises merged 1km MGRS cells with global coverage in parquet and cloud-optimized raster formats.

Dataset files (754 x .parquet)

Source Cooperative product page


FTW Inference Inputs

Input assets required for running inference with the FTW (Fields in the Wild) Baseline tool for field boundary detection. Contains raster TIFF files with near-global spatial coverage used as preprocessing or reference inputs to the inference pipeline.

Dataset files (4 x .tiff)

Source Cooperative product page

license: AI returned 'unknown' (README does not state a license). Verify with publisher before flipping to _state: reviewed.


FTW Inference output

This repo contains the results of inference run from Fields of The World models. Initially that will just be from https://fieldsofthe.world/ftw-inference-app/ but we may enable other tools to also output here. Generally we'll let anyone try out the models, but the output will be open by default, in this repository.


FTW Planet

PlanetScope SR (3m) variant of the FTW Benchmark dataset


FTW Tests

Just a place to do some online testing of FTW stuff


Global Fields of The World (FTW) Change


Confidence joined field boundaries

A set of global field boundaries with modeled confidence values.


USDA Crop Sequence Boundaries


Foursquare Open Source Places - Fused-partitioned

Foursquare Open Source Places is a global dataset of over 100 million points of interest including venues, businesses, and places worldwide. Each record includes name, address, contact details, social media handles, and hierarchical category labels. The dataset is distributed in GeoParquet format, spatially partitioned for efficient spatial querying.

points-of-interest global-coverage geoparquet land-use address-data

2024-11-19/places

latitude

[latlon] Latitude coordinate of the place

longitude

[latlon] Longitude coordinate of the place

locality

[placename] City or town name

region

[placename] State or province name

postcode

[zipcode] Postal code

[admin_code] Administrative region code or name

country

[admin_code] Country code

geometry

[geometry] Point geometry in EPSG:4326

5 files sampled for this table

2025-02-06/places

locality

[placename] City or town name

region

[placename] State or province name

postcode

[zipcode] Postal code

country

[admin_code] Country code

geometry

[geometry] Point geometry in EPSG:4326

bbox

STRUCT(xmin DOUBLE, ymin DOUBLE, xmax DOUBLE, ymax DOUBLE)

5 files sampled for this table

Source Cooperative product page

## README.md
[Foursquare Open Sources Place](https://opensource.foursquare.com/os-places/), with more than 100 million global places of interest.

This dataset is geospatially partitioned following Open GeoParquet standards, using [Fused](https://www.fused.io/).

# License

Copyright 2024 Foursquare Labs, Inc. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.
You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.

extent.spatial.bbox: omitted; spatial reference is categorical via [2024-11-19/places.latitude (latlon), 2024-11-19/places.longitude (latlon), 2024-11-19/places.locality (placename), 2024-11-19/places.region (placename), 2024-11-19/places.postcode (zipcode), 2024-11-19/places.admin_region (admin_code), 2024-11-19/places.country (admin_code), 2024-11-19/places.geometry (geometry), 2025-02-06/places.latitude (latlon), 2025-02-06/places.longitude (latlon), 2025-02-06/places.locality (placename), 2025-02-06/places.region (placename), 2025-02-06/places.postcode (zipcode), 2025-02-06/places.admin_region (admin_code), 2025-02-06/places.country (admin_code), 2025-02-06/places.geometry (geometry)]. Add bbox once the placename → bbox gazetteer covers it.


US Census and Crop Data Layers - Hex Fused-partitioned

Hex-indexed, re-partitioned Parquet versions of two US national datasets: the USDA Crop Data Layer (land use/land cover classifications for even years 2012–2024) and the US Census 2020 population and housing counts. Both are aggregated to H3 hex7 and hex8 spatial resolutions, enabling efficient tile-based spatial queries.

agriculture crop-classification land-use remote-sensing h3 census population demographics

hex7_2012.parquet

hex

[spatial_index] H3 resolution-7 cell index

hex7_2014.parquet

hex7_2016.parquet

hex7_2018.parquet

hex7_2020.parquet

hex7_2022.parquet

hex7_2024.parquet

hex8_2012.parquet

[spatial_index] H3 resolution-8 cell index

hex8_2014.parquet

hex8_2016.parquet

hex8_2018.parquet

hex8_2020.parquet

hex8_2022.parquet

hex8_2024.parquet

2020_partitioned_h7.parquet

state

[placename] US state name

county

[placename] US county name

2020_partitioned_h8.parquet

state

[placename] US state name

county

[placename] US county name

Source Cooperative product page

## README.md
# Hex Fused-partitioned datasets

## Overview

This repository contains re-paritioned and hex indexed version of:
- Crop Data Layer (`fused/hex/release_2025_04_beta/cdl/`)
- US Census data  (`fused/hex/release_2025_04_beta/census/`)

<img src="https://data.source.coop/fused/hex/docs/banner.png" alt="Banner image showing Census + CDL" width="100%"/>

## Quickstart

### Open Datasets in Fused

**Explore parquet metadata**

Access to this dataset is facilitated by the [Fused](https://www.fused.io/) platform. Browse the data on [Fused Workbench](https://www.fused.io/workbench/catalog/CDL_from_source_coop-d6f88129-73e3-469b-8ac4-7793c2acbeed).

**Query dataset directly**

## Details

[Fused](https://www.fused.io/) is providing these datasets as hex versions in both `hex7` and `hex8` resolutions.

You can explore the CDL data with one of our [public UDF](https://www.fused.io/workbench/catalog/CDL_from_source_coop-d6f88129-73e3-469b-8ac4-7793c2acbeed) at no cost.

**Crop Data Layer**

- Available for `[2012, 2014, 2016, 2018, 2020, 2022, 2024]`
- Available in `hex7` and `hex8` resolutions

**US Census Data**

- Available for 2020
- Available in `hex7` and `hex8` resolutions

## Original Dataset

**Crop Data Layer**

Data gathered from [the USDA CDL Download page](https://pdi.scinet.usda.gov/portal/apps/sites/#/cropcros/pages/download-data)

**US Census Data**

Data gathered from [2020 TIGER/Line Shapefile dataset on census.gov](https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.2020.html#list-tab-790442341)

## Citation

Cropland Data Layer from [USDA CroplandCros](https://pdi.scinet.usda.gov/portal/apps/sites/#/cropcros):

```
USDA National Agricultural Statistics Service Cropland Data Layer. 2025. Published crop-specific data layer [Online]. Available at https://croplandcros.scinet.usda.gov/ (accessed 2025; verified 2025). USDA-NASS, Washington, DC.
```

US Census from [data.census.org](https://data.census.gov/):

```
U.S. Census Bureau, "TIGER/Line Shapefiles", <https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.2020.html#list-tab-790442341>, accessed on April 29, 2025.
```

providers: role [processor, host] for 'Fused' is an AI guess — Processor role stated (hex repartitioning); host inferred from data being published under the fused/ account on Source Cooperative. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [hex7_2012.hex (spatial_index), hex7_2014.hex (spatial_index), hex7_2016.hex (spatial_index), hex7_2018.hex (spatial_index), hex7_2020.hex (spatial_index), hex7_2022.hex (spatial_index), hex7_2024.hex (spatial_index), hex8_2012.hex (spatial_index), hex8_2014.hex (spatial_index), hex8_2016.hex (spatial_index), hex8_2018.hex (spatial_index), hex8_2020.hex (spatial_index), hex8_2022.hex (spatial_index), hex8_2024.hex (spatial_index), 2020_partitioned_h7.hex (spatial_index), 2020_partitioned_h7.state (placename), 2020_partitioned_h7.county (placename), 2020_partitioned_h8.hex (spatial_index), 2020_partitioned_h8.state (placename), 2020_partitioned_h8.county (placename)]. Add bbox once a placename → bbox gazetteer is available.


Overture Maps - Fused-partitioned

The Overture Maps Foundation's global open vector map dataset, geospatially partitioned by Fused into approximately 24,000 GeoParquet files for cloud-optimized access. It covers administrative boundaries, building footprints, land use, places (points of interest), transportation networks, addresses, and water bodies at worldwide extent. Multiple versioned releases spanning February 2024 through March 2026 are available.

administrative-boundaries building-footprints land-use points-of-interest global-coverage geoparquet transportation-networks

2024-02-15-alpha-0

geometry

[geometry] WKB geometry in EPSG:4326

bbox

STRUCT(minx DOUBLE, maxx DOUBLE, miny DOUBLE, maxy DOUBLE)

names

STRUCT("primary" VARCHAR, common MAP(VARCHAR, VARCHAR), rules STRUCT(variant VARCHAR, "language" VARCHAR, "value" VARCHAR, "at" DOUBLE[], side VARCHAR)[])

[admin_code] ISO 3166-1 alpha-2 country code

isoSubCountryCode

[admin_code] ISO 3166-2 subdivision code

sources

STRUCT(property VARCHAR, dataset VARCHAR, recordId VARCHAR, confidence DOUBLE)[]

3 files sampled for this table

2024-03-12-alpha-0

geometry

[geometry] WKB geometry in EPSG:4326

sources

STRUCT(property VARCHAR, dataset VARCHAR, record_id VARCHAR, confidence DOUBLE)[]

3 files sampled for this table

2024-08-20-0

geometry

[geometry] WKB geometry in EPSG:4326

bbox

STRUCT(xmin FLOAT, xmax FLOAT, ymin FLOAT, ymax FLOAT)

sources

STRUCT(property VARCHAR, dataset VARCHAR, record_id VARCHAR, update_time VARCHAR, confidence DOUBLE)[]

categories

STRUCT("primary" VARCHAR, alternate VARCHAR[])

brand

STRUCT(wikidata VARCHAR, "names" STRUCT("primary" VARCHAR, common MAP(VARCHAR, VARCHAR), rules STRUCT(variant VARCHAR, "language" VARCHAR, "value" VARCHAR, "between" DOUBLE[], side VARCHAR)[]))

addresses

STRUCT(freeform VARCHAR, locality VARCHAR, postcode VARCHAR, region VARCHAR, country VARCHAR)[]

2 files sampled for this table

65.parquet

geometry

[geometry] WKB geometry in EPSG:4326

2024-10-23-0

[geometry] WKB geometry in EPSG:4326

country

[admin_code] Country code for the address record

postcode

[zipcode] Postal/ZIP code for the address

cartography

STRUCT(min_zoom INTEGER, max_zoom INTEGER, sort_key INTEGER)

3 files sampled for this table

50.parquet

geometry

[geometry] WKB geometry in EPSG:4326

country

[admin_code] Country code

region

[admin_code] Sub-national region code

67.parquet

geometry

[geometry] WKB geometry in EPSG:4326

75.parquet

[geometry] WKB geometry in EPSG:4326

country

[admin_code] Country code

cartography

STRUCT(prominence INTEGER, min_zoom INTEGER, max_zoom INTEGER, sort_key INTEGER)

region

[admin_code] Sub-national region code

hierarchies

STRUCT(division_id VARCHAR, subtype VARCHAR, "name" VARCHAR)[][]

STRUCT(division_id VARCHAR, subtype VARCHAR)[]

28.parquet

geometry

[geometry] WKB geometry in EPSG:4326

sources

STRUCT(property VARCHAR, dataset VARCHAR, record_id VARCHAR, update_time VARCHAR, confidence DOUBLE, "between" DOUBLE[])[]

names

STRUCT("primary" VARCHAR, common MAP(VARCHAR, VARCHAR), rules STRUCT(variant VARCHAR, "language" VARCHAR, perspectives STRUCT("mode" VARCHAR, countries VARCHAR[]), "value" VARCHAR, "between" DOUBLE[], side VARCHAR)[])

2025-09-24-0

geometry

[geometry] WKB geometry in EPSG:4326

sources

STRUCT(property VARCHAR, dataset VARCHAR, license VARCHAR, record_id VARCHAR, update_time VARCHAR, confidence DOUBLE, "between" DOUBLE[])[]

connectors

STRUCT(connector_id VARCHAR, "at" DOUBLE)[]

routes

STRUCT("name" VARCHAR, network VARCHAR, "ref" VARCHAR, symbol VARCHAR, wikidata VARCHAR, "between" DOUBLE[])[]

STRUCT("value" VARCHAR, "between" DOUBLE[])[]

STRUCT(access_type VARCHAR, "when" STRUCT(during VARCHAR, heading VARCHAR, "using" VARCHAR[], recognized VARCHAR[], "mode" VARCHAR[], vehicle STRUCT(dimension VARCHAR, comparison VARCHAR, "value" DOUBLE, unit VARCHAR)[]), "between" DOUBLE[])[]

STRUCT("value" INTEGER, "between" DOUBLE[])[]

destinations

STRUCT(labels STRUCT("value" VARCHAR, "type" VARCHAR)[], symbols VARCHAR[], from_connector_id VARCHAR, to_segment_id VARCHAR, to_connector_id VARCHAR, "when" STRUCT(heading VARCHAR), final_heading VARCHAR)[]

STRUCT("sequence" STRUCT(connector_id VARCHAR, segment_id VARCHAR)[], final_heading VARCHAR, "when" STRUCT(heading VARCHAR, during VARCHAR, "using" VARCHAR[], recognized VARCHAR[], "mode" VARCHAR[], vehicle STRUCT(dimension VARCHAR, comparison VARCHAR, "value" DOUBLE, unit VARCHAR)[]), "between" DOUBLE[])[]

STRUCT("values" VARCHAR[], "between" DOUBLE[])[]

STRUCT(min_speed STRUCT("value" INTEGER, unit VARCHAR), max_speed STRUCT("value" INTEGER, unit VARCHAR), is_max_speed_variable BOOLEAN, "when" STRUCT(during VARCHAR, heading VARCHAR, "using" VARCHAR[], recognized VARCHAR[], "mode" VARCHAR[], vehicle STRUCT(dimension VARCHAR, comparison VARCHAR, "value" DOUBLE, unit VARCHAR)[]), "between" DOUBLE[])[]

STRUCT("value" DOUBLE, "between" DOUBLE[])[]

2 files sampled for this table

2026-03-18-0

geometry

[geometry] WKB geometry in EPSG:4326

bbox

STRUCT(xmin DOUBLE, xmax DOUBLE, ymin DOUBLE, ymax DOUBLE)

country

[admin_code] Country code

region

[admin_code] Sub-national region code

4 files sampled for this table

Source Cooperative product page

## README.md
# Overture - Fused-partitioned

## Overview

This is the Overture dataset, geo-partitioned with Fused for convenient access. This dataset adheres to the [Overture Data Schema](https://docs.overturemaps.org/).

## Quickstart

Access to this dataset is facilitated by the [Fused](https://www.fused.io/) platform. Browse the data on [Fused Workbench](https://www.fused.io/public/UDF_Overture_Maps_Example).

To access using Python, the [Overture Maps Example](https://github.com/fusedio/udfs/tree/main/public/Overture_Maps_Example) UDF shows an example of how to access the data using Fused:

```python
import fused

udf = fused.load(
    "https://github.com/fusedio/udfs/tree/main/public/Overture_Maps_Example"
)

gdf = fused.run(udf, x=2622, y=6333, z=14)
gdf.plot()
```

You can query the data visually using our [notebook](https://github.com/fusedio/notebooks/blob/main/public/overture.ipynb).

## Partitioning

This dataset is geospatially partitioned following Open GeoParquet standards, using Fused.

For technical reasons, the release is named `2024-11-13-0` in S3 (note the dash instead of a period at the end).

Under each release, the data is divided into themes: [admins](https://docs.overturemaps.org/themes/admins/), [base](https://docs.overturemaps.org/themes/base/), [buildings](https://docs.overturemaps.org/themes/buildings/building), [places](https://docs.overturemaps.org/themes/places/place), and [transportation](https://docs.overturemaps.org/themes/transportation/). Each theme contains one or more tables, which is a collection of GeoParquet files (`0.parquet`, `1.parquet`, etc.), Parquet metadata (`_metadata`), and Parquet sample (`_sample`).

The older release, 2024-03-12-alpha-0, has the following themes:

- `theme=admins/type=administrative_boundary/` - Administrative boundaries such as country borders
- `theme=base/type=land_use/` - Land use
- `theme=base/type=water/` - Water boundaries
- `theme=buildings/type=building/` - Building footprints. Because of the size of this table, it is further subdivided into multiple parts. There is no semantic meaning to these parts, they are subdivisions to help partitioning.
    - `theme=buildings/type=building/part=0/`
    - `theme=buildings/type=building/part=1/`
    - `theme=buildings/type=building/part=2/`
    - `theme=buildings/type=building/part=3/`
    - `theme=buildings/type=building/part=4/`
- `theme=places/type=place/` - Places (POIs) such as restaurants and other businesses.
- `theme=transportation/type=connector/` - Points where transportation segments, below, are connected.
- `theme=transportation/type=segment/` - Line strings of how human movement is organized, such as roads and walkways.

There are older releases, 2024-03-12-alpha-0 and 2024-02-15-alpha-0 that have slightly different folder structures, where types are named in camel-case. (E.g. administrativeBoundary, landUse.)

## Original dataset

The original Overture datasets are downloadable from the [Overture website](https://overturemaps.org/download/). See the Overture [GitHub repo](https://github.com/OvertureMaps/data) for more information.

extent.spatial.bbox: omitted; spatial reference is categorical via [2024-02-15-alpha-0.geometry (geometry), 2024-02-15-alpha-0.isoCountryCodeAlpha2 (admin_code), 2024-02-15-alpha-0.isoSubCountryCode (admin_code), 2024-03-12-alpha-0.geometry (geometry), 2024-03-12-alpha-0.iso_country_code_alpha_2 (admin_code), 2024-03-12-alpha-0.iso_sub_country_code (admin_code), 2024-08-20-0.geometry (geometry), 2024-09-18-0.geometry (geometry), 2024-10-23-0.geometry (geometry), 2024-10-23-0.country (admin_code), 2024-10-23-0.postcode (zipcode), 2024-12-18-0.geometry (geometry), 2024-12-18-0.country (admin_code), 2024-12-18-0.region (admin_code), 2025-01-22-0.geometry (geometry), 2025-05-21-0.geometry (geometry), 2025-05-21-0.country (admin_code), 2025-05-21-0.region (admin_code), 2025-06-25-0.geometry (geometry), 2025-09-24-0.geometry (geometry), 2026-03-18-0.geometry (geometry), 2026-03-18-0.country (admin_code), 2026-03-18-0.region (admin_code)]. Add bbox once the placename → bbox gazetteer covers it.


World Bank Global Administrative Boundaries

High-resolution, World Bank-approved global administrative boundaries covering Admin 0 (countries), Admin 1 (states/provinces), and Admin 2 (counties/districts) levels, plus Non-Determined Legal Status Areas (NDLSA) and an ocean mask. A visualization line file for Admin 0 and supplementary data dictionaries joinable to Admin 1 and 2 features are included as separate downloads.

administrative-boundaries country-boundaries admin-regions world-bank land-use

Dataset files (12 x .geojson)

Source Cooperative product page


terrabit

Terrabit is a near-global raster (gridded) dataset spanning approximately -180° to 180° longitude and -83° to 84° latitude. It is stored as a Zarr archive composed of roughly 42,000 Parquet files totaling approximately 7.4 GB. Its specific subject matter, methodology, and provenance are undocumented.

land-use remote-sensing agriculture zarr global-coverage earth-observation

Dataset files (1 x .zarr)

Source Cooperative product page


Embeddings

Tile-level feature embeddings generated from machine learning models covering a geographic region in the southern United States.

embeddings machine-learning feature-extraction tile-based

Dataset files (127 x .parquet)

Source Cooperative product page


Geometries

A collection of 13 area-of-interest (AOI) geometries at the state or province level, spanning multiple continents across the Americas, Africa, and Asia. These polygons define the geographic extents over which model outputs have been generated.

Dataset files (13 x .geojson)

Source Cooperative product page


geovibes-datasets

A small geospatial dataset (one GeoTIFF and one GeoJSON file) generated using the Geovibes toolset, covering a region in the southeastern United States (approximately Alabama and adjacent states). No further description is available as the dataset has no README.

Dataset files (1 x .tif)

Source Cooperative product page

providers: role [producer, processor] for 'Geovibes' is an AI guess — Inferred as producer/processor from API description 'datasets generated using geovibes'; no explicit provider information stated.. Verify before flipping to _state: reviewed.


Search

A collection of geospatial image embeddings derived from satellite and aerial imagery over selected US areas of interest (Alabama, New Mexico), packaged as DuckDB databases bundled with nearest-neighbor similarity indices for efficient vector search. Each asset pairs per-tile geometries with high-dimensional embedding vectors produced by multiple foundation models including DINO ViT, EarthGenome SoftCon, Clay/NAIP, and Google Satellite Embeddings. Intended for similarity search and downstream machine learning tasks over Earth observation data.

embeddings machine-learning satellite-imagery foundation-model earth-observation remote-sensing similarity-search

alabama_clay_naip_v1.5_2019_2020_256_0_1_metadata_sorted.db

geometry

[geometry] Tile footprint geometry for each embedded image patch

metadata.db

metadata.db

metadata.db

metadata.db

metadata.db

metadata.db

new-mexico_google_satellite_embeddings_v1_2024_2025_25_0_10_metadata_sorted.db

experiments/cluster_storage_benchmark

6 files sampled for this table

Source Cooperative product page

providers: role [producer] for 'USDA National Agriculture Imagery Program (NAIP)' is an AI guess — Asset names reference NAIP imagery; assumed producer of underlying aerial imagery used for Clay/NAIP embeddings. Verify before flipping to _state: reviewed.

providers: role [producer] for 'Google' is an AI guess — Asset names reference Google Satellite Embeddings v1; assumed producer of underlying satellite imagery and/or embeddings. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [USA/alabama/2019-01-01-2020-01-01.geometry (geometry), USA/alabama/2019-01-01-2020-01-01/httpfs/alabama_clay_naip_v1.5_2019_2020_256_0_1.geometry (geometry), USA/alabama/2024-01-01-2025-01-01/httpfs/alabama_dino_vit_small_patch16_224_2024_2025_32_16_10.geometry (geometry), USA/alabama/2024-01-01-2025-01-01/httpfs/alabama_earthgenome_softcon_2024_2025_32_16_10.geometry (geometry), USA/alabama/2024-01-01-2025-01-01/httpfs/alabama_google_satellite_embeddings_v1_2024_2025_25_0_10.geometry (geometry), USA/alabama/2024-01-01-2025-01-01/httpfs/alabama_quantized_dino_vit_small_patch16_224_2024_2025_32_16_10.geometry (geometry), USA/new-mexico/2024-01-01-2025-01-01/httpfs/new-mexico_google_satellite_embeddings_v1_2024_2025_25_0_10.geometry (geometry), USA/new-mexico/2024-01-01-2025-01-01.geometry (geometry), experiments/cluster_storage_benchmark.geometry (geometry)]. Add bbox once the placename → bbox gazetteer covers it.


Canopy Height and Above Ground Biomass

No Description Provided


National Building Height Dataset

A national building height dataset for the United States. It is a comprehensive building height dataset by combining the Microsoft Building Footprints dataset and USA Structures.

geospatial geoparquet geopackage building


National Surface Depressions

A national dataset of surface depressions delineated from 10-meter elevation data from the USGS 3D Elevation Program. The depressions were identified using level-set algorithms applied at the HU8 watershed scale and aggregated to HU2 scale. Two variants are provided: all depressions and non-floodplain depressions.

hydrology topography watershed digital-elevation-model

Dataset files (4288 x .parquet)

Source Cooperative product page

## README.md
---
jupytext:
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.15.2
kernelspec:
  display_name: geo
  language: python
  name: python3
---

# National Surface Depressions

[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/source-coop-readme/blob/main/depressions/README.ipynb)

## Description

This dataset represents the extent and location of surface depressions derived from the 10-m resolution dataset from the 3D Elevation Program ([3DEP](https://www.usgs.gov/3d-elevation-program)). The levet-set algorithm available through the [lidar](https://lidar.gishub.org) Python package was used the process the DEM and delineate surface depressions at the HU8 watershed scale.

## Reference

- Wu, Q., Lane, C. R., Wang, L., Vanderhoof, M. K., Christensen, J. R., & Liu, H. (2019). Efficient Delineation of Nested Depression Hierarchy in Digital Elevation Models for Hydrological Analysis Using Level‐Set Method. JAWRA Journal of the American Water Resources Association , 55(2), 354-368. https://doi.org/10.1111/1752-1688.12689

## Environment setup

First, create a conda environment with the required packages:

```bash
conda create -n gdal python=3.11
conda activate gdal
conda install -c conda-forge mamba
mamba install -c conda-forge libgdal-arrow-parquet gdal leafmap
pip install lonboard
```

If you are using Google Colab, you can uncomment the following to install the packages and restart the runtime after installation.

```{code-cell} ipython3
# %pip install leafmap lonboard
```

## Data access

The dataset was derived from the 10-m resolution dataset from the 3D Elevation Program ([3DEP](https://www.usgs.gov/3d-elevation-program)) at HU8 watershed scale. The results were then merged at the HU2 watershed scale. Below is a map of the National Hydrography Dataset ([NHD](https://www.usgs.gov/media/images/watershed-boundary-dataset-structure-visualization)) watershed boundary structure.

![](https://i.imgur.com/UFivxid.png)

The script below can be used to access the data using [DuckDB](https://duckdb.org). The script uses the [duckdb](https://duckdb.org) Python package.

```{code-cell} ipython3
import duckdb

con = duckdb.connect()
con.install_extension("spatial")
con.load_extension("spatial")

hu2 = "06"    # Change to the HU2 of your choice
url = f"https://data.source.coop/giswqs/depressions/all_dep/HU2_{hu2}.parquet"
con.sql(f"SELECT * EXCLUDE geometry, ST_GeomFromWKB(geometry) as geometry FROM '{url}'")
```

Find out the total number non-floodplain wetlands in the selected watershed:

```{code-cell} ipython3
con.sql(f"SELECT COUNT(*) FROM '{url}'")
```

Alternatively, you can use the aws cli to access the data directly from the S3 bucket:

```bash
aws s3 ls s3://us-west-2.opendata.source.coop/giswqs/depressions/all_dep/
```

## Data visualization

To visualize the data, you can use the [leafmap](https://leafmap.org) Python package with the [lonboard](https://github.com/developmentseed/lonboard) backend. The script below shows how to visualize the data.

```{code-cell} ipython3
import leafmap

hu2 = "06"    # Change to the HU2 of your choice
url = f"https://data.source.coop/giswqs/depressions/all_dep/HU2_{hu2}.parquet"
gdf = leafmap.read_parquet(url, return_type='gdf', src_crs='EPSG:5070', dst_crs='EPSG:4326')
# leafmap.view_vector(gdf, get_fill_color=[0, 0, 255, 128])
```

## Data analysis

The depression dataset has two variations: `all_dep` and `nfp_dep`. The `all_dep` dataset contains all depressions, while the `nfp` dataset contains non-floodplain depressions.

### All depressions

Find out the total number of surface depression in the contiguous United States (CONUS):

```{code-cell} ipython3
con.sql(f"""
SELECT COUNT(*) AS Count
FROM 's3://us-west-2.opendata.source.coop/giswqs/depressions/all_dep/*.parquet'
""")
```

Calculate some descriptive statistics of all surface depressions:

```{code-cell} ipython3
con.sql(f"""
SELECT sum(area) as total_area, median(area) AS median_area, median(volume) AS median_volume, median(avg_depth) AS median_depth
FROM 's3://us-west-2.opendata.source.coop/giswqs/depressions/all_dep/*.parquet'
""")
```

### Non-floodplain depressions

Find out the total number non-floodplain depressions in the contiguous United States (CONUS):

```{code-cell} ipython3
con.sql(f"""
SELECT COUNT(*) AS Count
FROM 's3://us-west-2.opendata.source.coop/giswqs/depressions/nfp_dep/*.parquet'
""")
```

Calculate some descriptive statistics of non-floodplain surface depressions:

```{code-cell} ipython3
con.sql(f"""
SELECT sum(area) as total_area, median(area) AS median_area, median(volume) AS median_volume, median(avg_depth) AS median_depth
FROM 's3://us-west-2.opendata.source.coop/giswqs/depressions/nfp_dep/*.parquet'
""")
```


Non-Floodplain Wetlands

Geographically isolated wetlands (non-floodplain wetlands) of the conterminous United States, identified and mapped based on their geographic isolation from NHD aquatic systems and coastal features.

wetlands biodiversity conservation hydrology ecosystem environmental-monitoring

U.S. Environmental Protection Agency (EPA)

Dataset files (97 x .parquet)

Source Cooperative product page

## README.md
---
jupytext:
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.16.2
kernelspec:
  display_name: Python 3
  language: python
  name: python3
---

# Non-Floodplain Wetlands

[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/source-coop-readme/blob/main/giw/README.ipynb)

## Introduction

The datasets represent the extent and approximate location of [Geographically Isolated Wetlands](https://catalog.data.gov/dataset/geographically-isolated-wetlands-non-floodplain-wetlands-of-the-conterminous-united-states1) (GIWs), also known as non-floodplain wetlands (NFWs), in the conterminous United States. National Wetlands Inventory (NWI) lacustrine systems and palustrine wetlands were determined to be “isolated” based on their geographic location (i.e., unconnected, based on a distance measure, to specific classes of NHD aquatic systems). GIWs were here considered geographically isolated when they were outside of 10 meters from select NHD lines and polygons or were not adjacent to NWI Riverine or Estuarine wetlands and (where applicable) outside of 10 meters from a coastline (e.g., oceans or Great Lakes).

The datasets have two versions:

- [Version 1](https://beta.source.coop/giswqs/giw/wetlands) (46.37 GB): The original Lane & D'Amico (2016) dataset with the original attributes. We call this dataset **NFW**.
- [Version 2](https://beta.source.coop/giswqs/giw/wetlands_v2) (90.18 GB): The original Lane & D'Amico (2016) dataset with additional attributes from the 10-m resolution depression dataset. We call this dataset **Wetland Depressions**.

## Reference

- Lane, C. R., & D'Amico, E. (2016). Identification of putative geographically isolated wetlands of the conterminous United States. _JAWRA Journal of the American Water Resources Association_, 52(3), 705-722. https://doi.org/10.1111/1752-1688.12421

## Environment setup

First, create a conda environment with the required packages:

```bash
conda create -n gdal python=3.11
conda activate gdal
conda install -c conda-forge mamba
mamba install -c conda-forge libgdal-arrow-parquet gdal leafmap lonboard
```

If you are using Google Colab, you can uncomment the following to install the packages and restart the runtime after installation.

```{code-cell} ipython3
# %pip install leafmap lonboard
```

## Data download

Click on this [link](https://gaftp.epa.gov/EPADataCommons/ORD/CONUS_NFWs/Geographically_Isolated_Wetlands_of_ConterminousUnitedStates.gdb.zip) to download the data to your computer and unzip it.

## Data conversion

The script below was used to convert the data from the original Geodatabase format to [Parquet](https://parquet.apache.org) format. The script uses the [leafmap](https://leafmap.org) Python package.

```{code-cell} ipython3
import leafmap
```

```{code-cell} ipython3
gdb = "Geographically_Isolated_Wetlands_of_ConterminousUnitedStates.gdb"
# leafmap.gdb_to_vector(gdb, ".", gdal_driver="Parquet")
```

The total file size of the Geodatabase files is 4.4 GB. The total file size of the Parquet files is 46.37 GB.

## Data access

The script below can be used to access the data using [DuckDB](https://duckdb.org). The script uses the [duckdb](https://duckdb.org) Python package.

```{code-cell} ipython3
import duckdb

con = duckdb.connect()
con.install_extension("spatial")
con.load_extension("spatial")

state = "IA"  # Change to the US State of your choice
url = f"https://data.source.coop/giswqs/giw/wetlands/{state}_IW.parquet"
# con.sql(f"SELECT * EXCLUDE geometry, ST_GeomFromWKB(geometry) as geometry FROM '{url}'")
```

Find out the total number non-floodplain wetlands in the selected state:

```{code-cell} ipython3
con.sql(f"SELECT COUNT(*) FROM '{url}'")
```

Alternatively, you can use the aws cli to access the data directly from the S3 bucket:

```bash
aws s3 ls s3://us-west-2.opendata.source.coop/giswqs/giw/wetlands/
```

## Data visualization

To visualize the data, you can use the [leafmap](https://leafmap.org) Python package with the [lonboard](https://github.com/developmentseed/lonboard) backend. The script below shows how to visualize the data.

```{code-cell} ipython3
import leafmap

state = "DC"  # Change to the US State of your choice
url = f"https://data.source.coop/giswqs/giw/wetlands/{state}_IW.parquet"
gdf = leafmap.read_parquet(
    url, return_type="gdf", src_crs="EPSG:5070", dst_crs="EPSG:4326"
)
leafmap.view_vector(gdf, get_fill_color=[0, 0, 255, 128])
```

## Data analysis

Find out the total number of NFWs in CONUS:

```{code-cell} ipython3
con.sql(
    f"""
SELECT COUNT(*) AS Count
FROM 's3://us-west-2.opendata.source.coop/giswqs/giw/wetlands/*.parquet'
"""
)
```

The total number of NFWs in CONUS is 8,380,620.

+++

Find out the number of NFWs in each state and order them by the number of NFWs:

```{code-cell} ipython3
giw_count_df = con.sql(
    f"""
SELECT inState AS State, COUNT(*) AS Count
FROM 's3://us-west-2.opendata.source.coop/giswqs/giw/wetlands/*.parquet'
GROUP BY inState
ORDER BY COUNT(*) DESC;
"""
).df()
giw_count_df.head()
```

Create a pie chart showing the number of NFWs in each state:

```{code-cell} ipython3
leafmap.pie_chart(
    giw_count_df,
    "State",
    "Count",
    height=700,
    title="Number of Non-Floodplain Wetlands (NFWs) by State",
)
```

![](https://i.imgur.com/SNpVzcC.png)

Create a bar chart showing the number of NFWs in each state:

```{code-cell} ipython3
leafmap.bar_chart(
    giw_count_df, "State", "Count", title="Number of Non-Floodplain Wetlands (NFWs) by State"
)
```

![](https://i.imgur.com/19gxoBE.png)

Calculate the total area of NFWs in each state and order them by the area of wetlands:

```{code-cell} ipython3
sum_df = con.sql(
    f"""
SELECT inState AS State, Sum(hectares) AS Hectares
FROM 's3://us-west-2.opendata.source.coop/giswqs/giw/wetlands/*.parquet'
GROUP BY inState
ORDER BY Sum(hectares) DESC;
"""
).df()
sum_df.head()
```

Create a pie chart showing the total area of NFWs in each state:

```{code-cell} ipython3
leafmap.pie_chart(
    sum_df,
    "State",
    "Hectares",
    height=700,
    title="Percentage Area of Non-Floodplain Wetlands (NFWs) by State",
)
```

![](https://i.imgur.com/dayPfZY.png)

Create a pie chart showing the total area of NFWs in each state:

```{code-cell} ipython3
leafmap.bar_chart(
    sum_df, "State", "Hectares", title="Total Area of Non-Floodplain Wetlands (NFWs) by State"
)
```

```{code-cell} ipython3
median_df = con.sql(
    f"""
SELECT inState AS State, median(hectares)*10000 AS Meters
FROM 's3://us-west-2.opendata.source.coop/giswqs/giw/wetlands/*.parquet'
GROUP BY inState
ORDER BY median(hectares) DESC;
"""
).df()
median_df.head(10)
```

Create a bar chart showing the median area of NFWs in each state:

```{code-cell} ipython3
leafmap.bar_chart(
    median_df,
    "State",
    "Meters",
    title="Median Area of Non-Floodplain Wetlands (NFWs) by State",
)
```

![](https://i.imgur.com/OZahOKx.png)

+++

Calculate the number of NFWs intersecting surface depressions (i.e., wetland depressions) in each state:

```{code-cell} ipython3
giw_dep_count_stat = con.sql(
    f"""
SELECT inState AS State, COUNT(DISTINCT Final_ID) AS Count
FROM 's3://us-west-2.opendata.source.coop/giswqs/giw/wetlands_v2/*.parquet'
WHERE area IS NOT NULL
GROUP BY inState
ORDER BY COUNT(*) DESC;
"""
).df()
giw_dep_count_stat.head()
```

Merge the NFW table with the wetland depressions table:

```{code-cell} ipython3
merged_df = giw_count_df.merge(giw_dep_count_stat, on="State")
merged_df.columns = ["State", "GIW_Count", "DEP_Count"]
merged_df["Percent"] = merged_df["DEP_Count"] / merged_df["GIW_Count"] * 100
merged_df.head()
```

Compare the number of NFWs and wetland depressions in each state:

```{code-cell} ipython3
leafmap.bar_chart(
    merged_df,
    "State",
    ["GIW_Count", "DEP_Count"],
    title="Number of NFWs and Wetland Depressions by State",
)
```

![](https://i.imgur.com/BsPdZ82.png)

+++

Compute the statistics of the area of NFWs and wetland depressions in each state:

```{code-cell} ipython3
giw_dep_count_stat = con.sql(
    f"""
SELECT inState AS State, COUNT(DISTINCT Final_ID) AS Count,
SUM(area) / 1e6 AS Total_Area_km2,
SUM(volume) / 1e9 AS Total_Volume_km3,
Median(area) AS Median_Area_m2,
Median(volume) AS Median_Volume_m3
FROM 's3://us-west-2.opendata.source.coop/giswqs/giw/wetlands_v2/*.parquet'
WHERE area IS NOT NULL
GROUP BY inState
ORDER BY COUNT(*) DESC;
"""
).df()
giw_dep_count_stat.head(10)
```

The median size of wetland depressions in each state:

```{code-cell} ipython3
leafmap.bar_chart(
    giw_dep_count_stat[giw_dep_count_stat['State'] != "NV"],
    "State",
    "Median_Area_m2",
    title="The Median Size of Maximum Depression Area by State",
)
```

![](https://i.imgur.com/nkD0NSM.png)

+++

The median maximum storage volume of wetland depressions in each state:

```{code-cell} ipython3
leafmap.bar_chart(
    giw_dep_count_stat,
    "State",
    "Median_Volume_m3",
    title="The Median Maximum Storage Volume of Wetlands Depressions by State",
)
```

![](https://i.imgur.com/OLohMJ9.png)


National Wetlands Inventory

Comprehensive inventory of US wetlands with detailed information on their abundance, characteristics, and geographic distribution. The dataset has been converted from the original Geodatabase format to cloud-optimized Parquet files for efficient access and analysis.

wetlands biodiversity water conservation land-use remote-sensing

Dataset files (270 x .parquet)

Source Cooperative product page

## README.md
---
jupytext:
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.15.2
---

# National Wetlands Inventory

[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/source-coop-readme/blob/main/nwi/README.ipynb)

## Description

This dataset is a copy of the [National Wetlands Inventory](https://www.fws.gov/program/national-wetlands-inventory), offering the data in more GIS-friendly and [cloud-native geospatial](https://cloudnativegeo.org) formats. The original dataset is distributed as zipped Geodatabase files, and is available for download from [here](https://www.fws.gov/program/national-wetlands-inventory/download-state-wetlands-data).

## Data download

The script below was used to download the data from the National Wetlands Inventory in Geodatabase format from [here](https://www.fws.gov/program/national-wetlands-inventory/download-state-wetlands-data). The script uses the [leafmap](https://leafmap.org) Python package.

First, create a conda environment with the required packages:

```bash
conda create -n gdal python=3.11
conda activate gdal
conda install -c conda-forge mamba
mamba install -c conda-forge libgdal-arrow-parquet gdal leafmap
```

If you are using Google Colab, you can install the packages as follows:

```{code-cell} ipython3
# %pip install leafmap lonboard==0.3.0
```

Then, run the script below:

```{code-cell} ipython3
import leafmap
import pandas as pd

url = 'https://open.gishub.org/data/us/us_states.csv'
df = pd.read_csv(url)
ids = df['id'].tolist()
ids.sort()
urls = [f"https://documentst.ecosphere.fws.gov/wetlands/data/State-Downloads/{id}_geodatabase_wetlands.zip" for id in ids]
leafmap.download_files(urls, out_dir='.', unzip=True)
```

## Data conversion

The script below was used to convert the data from the original Geodatabase format to [Parquet](https://parquet.apache.org) format. The script uses the [leafmap](https://leafmap.org) Python package.

```{code-cell} ipython3
import leafmap
import pandas as pd

url = 'https://open.gishub.org/data/us/us_states.csv'
df = pd.read_csv(url)
ids = df['id'].tolist()

for index, state in enumerate(ids):
    print(f'Processing {state} ({index+1}/{len(ids)})')
    gdb = f"{state}_geodatabase_wetlands.gdb/"
    layer_name = f'{state}_Wetlands'
    leafmap.gdb_to_vector(gdb, ".", gdal_driver="Parquet", layers=[layer_name])
```

The total file size of the Geodatabase files is 32.5 GB. The total file size of the Parquet files is 75.8 GB.

## Data access

The script below can be used to access the data using [DuckDB](https://duckdb.org). The script uses the [duckdb](https://duckdb.org) Python package.

```{code-cell} ipython3
import duckdb

con = duckdb.connect()
con.install_extension("spatial")
con.load_extension("spatial")

state = "DC"    # Change to the US State of your choice
url = f"https://data.source.coop/giswqs/nwi/wetlands/{state}_Wetlands.parquet"
con.sql(f"SELECT * EXCLUDE geometry, ST_GeomFromWKB(geometry) FROM '{url}'")
```

Alternatively, you can use the aws cli to access the data directly from the S3 bucket:

```bash
aws s3 ls s3://us-west-2.opendata.source.coop/giswqs/nwi/wetlands/
```

## Data visualization

To visualize the data, you can use the [leafmap](https://leafmap.org) Python package with the [lonboard](https://github.com/developmentseed/lonboard) backend. The script below shows how to visualize the data.

```{code-cell} ipython3
import leafmap

state = "DC"   # Change to the US State of your choice
url = f"https://data.source.coop/giswqs/nwi/wetlands/{state}_Wetlands.parquet"
gdf = leafmap.read_parquet(url, return_type='gdf', src_crs='EPSG:5070', dst_crs='EPSG:4326')
leafmap.view_vector(gdf, get_fill_color=[0, 0, 255, 128])
```

![vector](https://i.imgur.com/HRtpiVd.png)

Alternatively, you can specify a color map to visualize the data.

```{code-cell} ipython3
color_map =  {
        "Freshwater Forested/Shrub Wetland": (0, 136, 55),
        "Freshwater Emergent Wetland": (127, 195, 28),
        "Freshwater Pond": (104, 140, 192),
        "Estuarine and Marine Wetland": (102, 194, 165),
        "Riverine": (1, 144, 191),
        "Lake": (19, 0, 124),
        "Estuarine and Marine Deepwater": (0, 124, 136),
        "Other": (178, 134, 86),
    }
leafmap.view_vector(gdf, color_column='WETLAND_TYPE', color_map=color_map, opacity=0.5)
```

![vector-color](https://i.imgur.com/Ejh8hK6.png)

Display a legend for the data.

```{code-cell} ipython3
leafmap.Legend(title="Wetland Type", legend_dict=color_map)
```

![legend](https://i.imgur.com/fxzHHFN.png)

## Data analysis

Find out the total number of wetlands in the United States by aggregating the 51 parquet files.

```{code-cell} ipython3
con.sql(f"""
SELECT COUNT(*) AS Count
FROM 's3://us-west-2.opendata.source.coop/giswqs/nwi/wetlands/*.parquet'
""")
```

Find out the number of wetlands in each state. Note that the NWI datasets do not contain a field for state names. The `filename` argument can be used to add an extra `filename` column to the result that indicates which row came from which file.

```{code-cell} ipython3
df = con.sql(f"""
SELECT filename, COUNT(*) AS Count
FROM read_parquet('s3://us-west-2.opendata.source.coop/giswqs/nwi/wetlands/*.parquet', filename=true)
GROUP BY filename
ORDER BY COUNT(*) DESC;
""").df()
df.head()
```

Inspect the list of filenames.

```{code-cell} ipython3
df['filename'].tolist()
```

Create a `State` column based on the `filename` column by extracting the state name from the filename.

```{code-cell} ipython3
count_df = con.sql(f"""
SELECT SUBSTRING(filename, LENGTH(filename) - 18, 2) AS State, COUNT(*) AS Count
FROM read_parquet('s3://us-west-2.opendata.source.coop/giswqs/nwi/wetlands/*.parquet', filename=true)
GROUP BY State
ORDER BY COUNT(*) DESC;
""").df()
count_df.head(10)
```

Create a `wetlands` table from the DataFrame above.

```{code-cell} ipython3
con.sql("CREATE OR REPLACE TABLE wetlands AS FROM count_df")
con.sql("FROM wetlands")
```

To visualize the data on the map, we need a GeoDataFrame. Let's create a `states` table from the [us_states.parquet](https://open.gishub.org/data/us/us_states.parquet) file.

```{code-cell} ipython3
url = 'https://open.gishub.org/data/us/us_states.parquet'
con.sql(
    f"""
CREATE OR REPLACE TABLE states AS
SELECT * EXCLUDE geometry, ST_GeomFromWKB(geometry)
AS geometry FROM '{url}'
"""
)
con.sql("FROM states")
```

Join the `wetlands` table with the `states` table.

```{code-cell} ipython3
con.sql("""
SELECT * FROM states INNER JOIN wetlands ON states.id = wetlands.State
""")
```

Export the joined table as a Pandas DataFrame.

```{code-cell} ipython3
df = con.sql("""
SELECT name, State, Count, ST_AsText(geometry) as geometry
FROM states INNER JOIN wetlands ON states.id = wetlands.State
""").df()
df.head()
```

Convert the Pandas DataFrame to a GeoDataFrame.

```{code-cell} ipython3
gdf = leafmap.df_to_gdf(df, src_crs="EPSG:4326")
```

Visualize the data on the map.

```{code-cell} ipython3
m = leafmap.Map()
m.add_data(
    gdf, column='Count', scheme='Quantiles', cmap='Greens', legend_title='Wetland Count'
)
m
```

![](https://i.imgur.com/x9nJWZR.png)

Create a pie chart to show the percentage of wetlands in each state.

```{code-cell} ipython3
leafmap.pie_chart(count_df, 'State', 'Count', height=800, title='Number of Wetlands by State')
```

![](https://i.imgur.com/EQFZW4x.png)

Create a bar chart to show the number of wetlands in each state.

```{code-cell} ipython3
leafmap.bar_chart(count_df, 'State', 'Count', title='Number of Wetlands by State')
```

![](https://i.imgur.com/dNjh9lp.png)

Calculate the total area of wetlands in the United States. It takes about 3 minutes to run this query. Please be patient.

```{code-cell} ipython3
con.sql(f"""
SELECT SUM(Shape_Area) /  1000000 AS Area_SqKm
FROM 's3://us-west-2.opendata.source.coop/giswqs/nwi/wetlands/*.parquet'
""")
```

Calculate the total area of wetlands in each state. It takes about 3 minutes to run this query. Please be patient.

```{code-cell} ipython3
area_df = con.sql(f"""
SELECT SUBSTRING(filename, LENGTH(filename) - 18, 2) AS State, SUM(Shape_Area) /  1000000 AS Area_SqKm
FROM read_parquet('s3://us-west-2.opendata.source.coop/giswqs/nwi/wetlands/*.parquet', filename=true)
GROUP BY State
ORDER BY COUNT(*) DESC;
""").df()
area_df.head(10)
```

Create a pie chart to show the percentage of wetlands in each state.

```{code-cell} ipython3
leafmap.pie_chart(area_df, 'State', 'Area_SqKm', height=900, title='Wetland Area by State')
```

![](https://i.imgur.com/tIy2fLt.png)

Create a bar chart to show the wetland area in each state.

```{code-cell} ipython3
leafmap.bar_chart(area_df, 'State', 'Area_SqKm', title='Wetland Area by State')
```

![](https://i.imgur.com/EyJQZNP.png)


Data for Open Geospatial Solutions

A collection of open geospatial raster and vector datasets covering North America, comprising approximately 1,460 GeoTIFF rasters alongside PMTiles, Parquet, CSV, and GeoJSON assets totaling roughly 8.5 GB. Thematic content spans land use and remote sensing applications across a spatial extent from roughly Alaska/western Canada to the Caribbean and eastern North America. Specific subject matter is undocumented due to the absence of a README.

land-use remote-sensing cloud-optimized-geotiff pmtiles geoparquet earth-observation

Dataset files (1495 x .tif)

Source Cooperative product page


Playa Wetlands

Geospatial dataset of playa wetlands—shallow, seasonal water bodies characteristic of arid and semi-arid regions. Contains over 5 million features across multiple file formats including parquet, GeoPackage, and PMTiles.

wetlands biodiversity land-use water-bodies arid-lands

Dataset files (12 x .parquet)

Source Cooperative product page


Tennessee Historical Imagery

Historical aerial imagery of Tennessee acquired between 1997 and 2006, covering 94 of the state's 95 counties (Knox County excluded). Images are provided as Cloud Optimized GeoTIFFs converted from the original MrSID files sourced from Tennessee's state mapping portal.

remote-sensing historical-imagery aerial-imagery tennessee cloud-optimized-geotiff

Dataset files (89 x .tif)

Source Cooperative product page

providers: role [producer, licensor] for 'Tennessee State Mapping Program (tnmap.maps.arcgis.com)' is an AI guess — Assumed producer/licensor as the originating state GIS portal; specific agency (e.g. TDOT or TNGIC) not stated in the source description.. Verify before flipping to _state: reviewed.


EOTA-PDF-Archive

All PDF documents found in the 2008-2024 End of Term Archive web crawls.


data.gov metadata snapshots

Snapshots of metadata from the data.gov API.


DuckDb Test

Test for browsing a database with https://github.com/harvard-lil/duckdb-test


Federal github repositories

Snapshots of United States federal Github repositories created with https://github.com/harvard-lil/gitspoke/


Archive of Data.gov

This collection includes BagIt archives of every entry on Data.gov, including API metadata for each dataset.


Smithsonian Open Access Archive

This repository includes approximately 784 terabytes (8.7 million files) of public domain data from the Smithsonian Institution's Open Access collection. Sourced from more than 20 libraries, museums, and research centers across the Smithsonian, this archive is updated weekly.


[STAGING] Archive of data.gov

Staging area for Data.gov archive and associated projects


Google Open Buildings

Building footprints derived from Google's machine learning models applied to satellite imagery, covering approximately 20 million buildings. Each feature includes the building polygon geometry, footprint area in square meters, and a model confidence score. Distributed in cloud-native GeoParquet 1.1 format with support for spatial filtering.

building-footprints remote-sensing satellite-imagery machine-learning geoparquet earth-observation

buildings

[geometry] Building polygon geometry encoded as GeoParquet geometry column

10 files sampled for this table

Source Cooperative product page

extent.spatial.bbox: omitted; spatial reference is categorical via [buildings.geometry (geometry)]. Add bbox once the placename → bbox gazetteer covers it.


HDX HAPI Cloud-Native

Parquet snapshots of the HDX Humanitarian API (HDX HAPI), operated by OCHA, aggregating standardized humanitarian data from multiple organizations across crisis-affected countries. Themes include affected people (refugees, IDPs, humanitarian needs), food security and prices, conflict events, poverty rates, baseline population, climate/rainfall, and operational presence of humanitarian organizations. Data is normalized to common administrative boundary codes to enable cross-source interoperability and automated humanitarian workflows.

humanitarian hapi hdx hxl ocha parquet food-security demographics administrative-boundaries displacement conflict precipitation

UN OCHA HDX Humanitarian API

The HDX Humanitarian API (HAPI) provides standardized humanitarian indicators from multiple sources to facilitate automated workflows and visualizations supporting humanitarian decision-making.

UN Office for the Coordination of Humanitarian Affairs (OCHA)

affected-people/humanitarian-needs

[admin_code] ISO country-level location code (pattern repeats across all assets)

[placename] Country name (pattern repeats across all assets)

[admin_code] Admin level 1 region code (pattern repeats across all assets)

[placename] Admin level 1 region name (pattern repeats across all assets)

[admin_code] Admin level 2 region code (pattern repeats across all assets)

[placename] Admin level 2 region name (pattern repeats across all assets)

3 files sampled for this table

affected-people/idps

3 files sampled for this table

affected-people

2 files sampled for this table

climate/rainfall

2 files sampled for this table

coordination-context/conflict-events

3 files sampled for this table

coordination-context

2 files sampled for this table

coordination-context/operational-presence

2 files sampled for this table

food-security-nutrition-poverty/food-prices-market-monitor

lat

[latlon] Latitude of the market location

lon

[latlon] Longitude of the market location

2 files sampled for this table

food-security-nutrition-poverty/food-security

3 files sampled for this table

food-security-nutrition-poverty/poverty-rate

2 files sampled for this table

geography-infrastructure/baseline-population

3 files sampled for this table

metadata

10 files sampled for this table

metadata/data-availability

3 files sampled for this table

metadata/wfp-market

lat

[latlon] Latitude of the WFP market reference point

lon

[latlon] Longitude of the WFP market reference point

2 files sampled for this table

Source Cooperative product page

HDX HAPI — per-dataset license information

HDX Humanitarian API

HDX HAPI documentation

providers: role [producer] for 'HDX contributing organizations (WFP, UNHCR, IPC, and others)' is an AI guess — Multiple humanitarian organizations contribute source datasets through HDX; specific producers per sub-dataset are catalogued in the metadata asset but not enumerated in the README.. Verify before flipping to _state: reviewed.


Microsoft Building Footprints

Microsoft Building Footprints is a global dataset of building outline polygons derived from satellite imagery using Microsoft's machine learning models. Each record includes the building geometry, an estimated height, a model confidence score, and a geographic region name. The dataset is distributed in cloud-native GeoParquet 1.1 format following OGC best practices.

building-footprints machine-learning remote-sensing satellite-imagery geoparquet global-coverage

geometry

[geometry] Building outline polygon geometry

RegionName

[placename] Geographic region name associated with the building

quadkey

[spatial_index] Bing Maps quadkey tile index used for spatial partitioning

Source Cooperative product page

extent.spatial.bbox: omitted; spatial reference is categorical via [part-00108-4feead82-d499-422b-94cb-c036c212127a.c000.geometry (geometry), part-00108-4feead82-d499-422b-94cb-c036c212127a.c000.RegionName (placename), part-00108-4feead82-d499-422b-94cb-c036c212127a.c000.quadkey (spatial_index)]. Add bbox once the placename → bbox gazetteer covers it.


OpenPOIs

OpenPOIs is a unified dataset of over 4.5 million points of interest across the United States, built by conflating OpenStreetMap and Overture Maps sources. Each POI includes model-estimated confidence scores (probability the POI currently exists), harmonized category labels, and source-level diagnostics linking back to OSM and Overture identifiers. Data are distributed as hive-partitioned Parquet files and PMTiles archives, versioned by release date.

points-of-interest openstreetmap administrative-boundaries pmtiles geoparquet conflation venue-data machine-learning

2026-04-23-v0/conflated-parquet

geometry

[geometry] WKB point geometry in EPSG:4326

geohash

[spatial_index] Geohash spatial index for the POI location

3 files sampled for this table

latest/osm-parquet

[placename] State name or abbreviation from OSM address tags

[zipcode] Postal/ZIP code from OSM address tags

[admin_code] Country code from OSM address tags

[placename] City name from OSM address tags

bbox

STRUCT(xmin DOUBLE, ymin DOUBLE, xmax DOUBLE, ymax DOUBLE)

5 files sampled for this table

Source Cooperative product page

## README.md
# OpenPOIs

A unified, open dataset for points of interest across the United States. Built from [OpenStreetMap](https://www.openstreetmap.org) and
[Overture Maps](https://overturemaps.org), with per-POI confidence scores
produced by the OpenPOIs turnover model.

- 🌐 **Interactive map:** <https://openpois.org/>
- 💻 **Source code:** <https://github.com/henryspatialanalysis/openpois>
- 📘 **Data license:** [Open Database License v.1.0](./LICENSE). For more details, see the [Open Data Commons](https://opendatacommons.org/licenses/odbl/1-0/).

## Repository layout

Each refresh writes a new versioned folder. Inside every version:

```
<YYYY-MM-DD-vN>/
├── README.md                     # version metadata (OSM date, Overture release, model)
├── osm-parquet/                  # OSM-only snapshot, hive-partitioned by primary_tag
├── osm-pmtiles/osm.pmtiles       # OSM snapshot as a single PMTiles archive
├── conflated-parquet/            # OSM × Overture conflated snapshot, hive-partitioned by shared_label
└── conflated-pmtiles/conflated.pmtiles
```

`latest/` is a server-side mirror of the most recently published version —
use it for live demos and tutorials, and pin a dated folder
(e.g. `2026-04-23-v0/`) when you need a stable, reproducible reference.

Browse all versions at
<https://source.coop/henryspatialanalysis/openpois>.

## What's in the conflated dataset

One row per real-world POI after matching OpenStreetMap features against
Overture Maps places. Key columns:

| Column | Description |
|---|---|
| `unified_id` | Stable ID for the conflated POI |
| `source` | `osm`, `overture`, or `both` |
| `osm_id`, `osm_type` | Source OSM feature (when present) |
| `overture_id` | Source Overture ID (when present) |
| `name`, `brand` | Preferred display names |
| `shared_label` | Harmonised category across the two source taxonomies |
| `conf_mean` | Model-estimated probability the POI currently exists (1 = exists, 0 = incorrect or stale) |
| `conf_lower`, `conf_upper` | 90% uncertainty interval for confidence score |
| `match_score`, `match_distance_m` | Diagnostics for the OSM × Overture link |
| `geometry` | WKB point (EPSG:4326) |

The `osm-parquet/` files contain the same OSM rows before conflation. This data retains the original OSM tags.

## Quickstart

Read directly from Source Cooperative's S3 mirror (no authentication):

- **pyarrow / GeoPandas** use `pyarrow.fs.S3FileSystem(anonymous=True)` and a
  bucket-qualified path (no scheme prefix).
- **DuckDB** uses an `s3://` URL plus an anonymous `SECRET` so its glob
  expansion works over the bucket listing.

Every example uses `VERSION = "latest"`; swap in a dated folder (e.g.
`"2026-04-23-v0"`) when you need a reproducible pin.

### Python: pyarrow

```python
import pyarrow.dataset as ds
import pyarrow.fs as pafs

BASE = "us-west-2.opendata.source.coop/henryspatialanalysis/openpois"
VERSION = "latest"   # or pin a specific dated folder, e.g. "2026-04-23-v0"

fs = pafs.S3FileSystem(anonymous = True, region = "us-west-2")
pois = ds.dataset(
    f"{BASE}/{VERSION}/conflated-parquet/",
    filesystem = fs,
    format = "parquet",
    partitioning = "hive",
)
print(pois.schema)
print(f"{pois.count_rows():,} POIs")
```

### Python: DuckDB

```python
import duckdb

BASE = "s3://us-west-2.opendata.source.coop/henryspatialanalysis/openpois"
VERSION = "latest"   # or pin a specific dated folder, e.g. "2026-04-23-v0"

con = duckdb.connect()
con.execute("INSTALL httpfs; LOAD httpfs;")
con.execute("""
    CREATE OR REPLACE SECRET srccoop (
        TYPE s3, PROVIDER config,
        REGION 'us-west-2', URL_STYLE 'path',
        KEY_ID '', SECRET ''
    );
""")
df = con.execute(f"""
    SELECT shared_label, COUNT(*) AS n
    FROM read_parquet('{BASE}/{VERSION}/conflated-parquet/**/*.parquet',
                      hive_partitioning = true)
    GROUP BY shared_label
    ORDER BY n DESC
    LIMIT 20
""").df()
print(df)
```

### Python: GeoPandas

```python
import geopandas as gpd
import pyarrow.fs as pafs

BASE = "us-west-2.opendata.source.coop/henryspatialanalysis/openpois"
VERSION = "latest"   # or pin a specific dated folder, e.g. "2026-04-23-v0"

fs = pafs.S3FileSystem(anonymous = True, region = "us-west-2")
# conflated-parquet is hive-partitioned by shared_label.
gdf = gpd.read_parquet(
    f"{BASE}/{VERSION}/conflated-parquet/shared_label=Cafe/part-0.parquet",
    filesystem = fs,
)
print(gdf.head())
```

### Browser / vector-tile map

The `*-pmtiles/*.pmtiles` archives can be loaded directly by any PMTiles
client (MapLibre + `pmtiles://`, OpenLayers + `ol-pmtiles`, etc.). See
`site/` in the GitHub repo for a working example.

The snippet below is a self-contained HTML page that renders the conflated
PMTiles over MapLibre, coloured by the model's `conf_mean`. Save it as
`openpois.html` and open it in a browser — no build step, no server needed.
PMTiles are authored at zoom 14, so zoom in past z14 to see points.

```html
<!doctype html>
<meta charset="utf-8" />
<title>OpenPOIs — conflated</title>
<link href="https://unpkg.com/maplibre-gl@4/dist/maplibre-gl.css" rel="stylesheet" />
<style>html, body, #map { height: 100%; margin: 0; }</style>
<div id="map"></div>
<script src="https://unpkg.com/maplibre-gl@4/dist/maplibre-gl.js"></script>
<script src="https://unpkg.com/pmtiles@3/dist/pmtiles.js"></script>
<script>
  const BASE = "https://data.source.coop/henryspatialanalysis/openpois";
  const VERSION = "latest";   // or pin a specific dated folder, e.g. "2026-04-23-v0"

  const protocol = new pmtiles.Protocol();
  maplibregl.addProtocol("pmtiles", protocol.tile);

  const map = new maplibregl.Map({
    container: "map",
    style: "https://tiles.openfreemap.org/styles/positron",
    center: [-73.9855, 40.758],   // Times Square
    zoom: 16,
  });

  map.on("load", () => {
    map.addSource("openpois", {
      type: "vector",
      url: `pmtiles://${BASE}/${VERSION}/conflated-pmtiles/conflated.pmtiles`,
      minzoom: 14,
    });
    map.addLayer({
      id: "openpois-points",
      type: "circle",
      source: "openpois",
      "source-layer": "conflated_pois",   // set by publish.pmtiles.conflated_layer_name
      paint: {
        "circle-radius": 4,
        "circle-stroke-width": 1,
        "circle-stroke-color": "#ffffff",
        // Red when stale (conf_mean ≈ 0), green when fresh (≈ 1).
        "circle-color": [
          "interpolate", ["linear"], ["get", "conf_mean"],
          0.0, "#d73027",
          0.3, "#fee08b",
          0.7, "#1a9850",
        ],
      },
    });
    map.on("click", "openpois-points", (e) => {
      const p = e.features[0].properties;
      new maplibregl.Popup()
        .setLngLat(e.lngLat)
        .setHTML(
          `<b>${p.name ?? "(no name)"}</b><br>` +
          `${p.shared_label} · source=${p.source}<br>` +
          `conf_mean = ${Number(p.conf_mean).toFixed(3)}`
        )
        .addTo(map);
    });
  });
</script>
```

## License & attribution

The OpenPOIs dataset is released under the [Open Database License (ODbL) v.1.0](./LICENSE). Any public use must credit OpenStreetMap contributors, the Overture Maps Foundation, and OpenPOIs. Any derivative database must be shared under the same license. See <https://www.openstreetmap.org/copyright> and <https://docs.overturemaps.org/attribution/> for upstream attribution
requirements.

## Citation

If you use this data in research, please cite:

> Henry Spatial Analysis (2026). *OpenPOIs: a unified, confidence-scored
> dataset of U.S. points of interest.* <https://openpois.henryspatialanalysis.com>

## Contact

Questions, bug reports, and contributions welcome via <https://github.com/henryspatialanalysis/openpois/issues>.

extent.spatial.bbox: omitted; spatial reference is categorical via [2026-04-23-v0/conflated-parquet.geometry (geometry), 2026-04-23-v0/conflated-parquet.geohash (spatial_index), latest/osm-parquet.geometry (geometry), latest/osm-parquet.geohash (spatial_index), latest/osm-parquet.addr:city (placename), latest/osm-parquet.addr:state (placename), latest/osm-parquet.addr:postcode (zipcode), latest/osm-parquet.addr:country (admin_code)]. Add bbox once the placename → bbox gazetteer covers it.


Japanese Vegetation Map (1/25,000)

This data is based on the Natural Environment Conservation Basic Survey conducted by the Ministry of the Environment's Biodiversity Center. The spatial scale of this data set is in 1/25,000. Original data was prepared as a series of Shapefile and publially avaiable for your download (http://gis.biodic.go.jp/webgis/sc-025.html?kind=vg67). We downloaded and merged all shapefile while fixing various schema anomality. We also translated categorical descriptions into English using ChatGPT4o.


IDI test source.coop #01

This is a test


Scout - Test repo

This is a test repository for project Scout.


Archive of Many Outdoor Scenes (AMOSv1)

A dataset containing hundreds of millions of images gathered from publicly available outdoor webcams.


test

test


GNDC-HiGLASS-LS20: Gap-free 20 m 5-Day LAI/FAPAR over China, 2018–2023

# GNDC-HiGLASS-LS20

GNDC-compressed version of **HiGLASS-LS20** — the first gap-free, 20 m, 5-day
LAI / FAPAR product over China (2018–2024), derived from integrated
Landsat-8/9 and Sentinel-2 Analysis Ready Data.

Files are stored as `.gndc` (a neural data cube format; see GeoNDC reference
below) instead of GeoTIFF to reduce size. **A decoder is required to read them.**

## Decode

```bash
pip install pygndc
```

Documentation: https://github.com/jianboqi/pygndc/blob/main/TUTORIAL.md

### Minimal example

```python
import pygndc

# Decode a .gndc file. The output is already in PHYSICAL values
# (no scale factor needs to be applied).
data = pygndc.read("HiGLASS_LS20_n018e108_2020_v01_LAI.gndc")

print(data.shape)   # (73, rows, cols) — 73 five-day composites
print(data[0])      # first composite (DOY 001), LAI in m² m⁻²
```

> Note: after decoding, values are already physical (LAI in m² m⁻²,
> FAPAR unitless). Do **not** multiply by the scale factors below — those
> apply only to the raw uint8 GeoTIFF source and are listed for reference.

## File naming

```
GNDC-HiGLASS_LS20_[lat][lon]_[year]_v01_LAI.gndc
GNDC-HiGLASS_LS20_[lat][lon]_[year]_v01_FAPAR.gndc
```

- Tiles are 1° × 1° (WGS84 / EPSG:4326), centred at integer degree + 0.5°
  (e.g. `n018e108` = 18.5°N, 108.5°E, covering ~18–19°N, 108–109°E).
- 73 bands per file = 5-day composites: DOY 001, 006, 011, …, 361
  (Jan 1, Jan 6, …, Dec 22).

## Physical values & ranges

| Variable | Range | Unit |
|----------|-------|------|
| LAI      | 0 – 10.0 | m² m⁻² |
| FAPAR    | 0 – 1.0  | unitless |

(For reference, the original uint8 GeoTIFF used scale factors LAI ×0.1 and
FAPAR ×0.004. pygndc returns decoded data already in the physical ranges above.)

## Extend to other regions (2019–2024)

Open-source pipeline (Google Colab):
https://colab.research.google.com/drive/1zGYbUkh4COoZqpWA2tfMLZHVfsGZ1nAo

## Citation

**For the GNDC compression format / decoder:**

Qi, J., M. Li, B. Jiang, Y. Chen, and Q. Wang (2026).
"GeoNDC: A Queryable Neural Data Cube for Planetary-Scale Earth Observation."
arXiv:2603.25037 [cs.CV]. https://arxiv.org/abs/2603.25037

```bibtex
@misc{qi2026geondc,
  title={GeoNDC: A Queryable Neural Data Cube for Planetary-Scale Earth Observation},
  author={Jianbo Qi and Mengyao Li and Baogui Jiang and Yidan Chen and Qiao Wang},
  year={2026},
  eprint={2603.25037},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2603.25037}
}
```

**For the underlying HiGLASS-LS20 product:**

Ma, H., Q. Wang, W. Li, Y. Chen, J. Xu, Y. Ma, J. Huang, and S. Liang (2025).
"The first gap-free 20 m 5-day LAI/FAPAR products over China (2018–2023) from
integrated Landsat-8/9 and Sentinel-2 Analysis Ready Data."
*Remote Sensing of Environment*, 331, 115048.
DOI: 10.1016/j.rse.2025.115048

## Contact

- Original product: Dr. Han Ma — mahan@hku.hk
- GNDC packaging & pygndc decoder: Jianbo Qi — jianboqi@bnu.edu.cn


Catalog of IDR images formatted as OME-NGFF

A mirror of OME-Zarr (OME-NGFF) formatted bioimaging files sourced from the Image Data Resource (IDR), a public repository of reference imaging datasets from peer-reviewed studies. The dataset contains Zarr-structured image data following the OME Next Generation File Format specification.

bioimaging microscopy ome-zarr ngff zarr image-data-resource

Dataset files (1 x .zarr)

Source Cooperative product page

providers: role [producer] for 'Image Data Resource (IDR)' is an AI guess — Assumed producer as IDR is the upstream source being mirrored; licensor role not stated. Verify before flipping to _state: reviewed.

providers: role [processor] for 'Open Microscopy Environment (OME)' is an AI guess — Assumed processor as OME maintains the ome-ngff-samples conversion pipeline referenced in the description. Verify before flipping to _state: reviewed.


Agri-environmental Semantic Segmentation of LUCAS landscape photos

This Dataset is a collection of street-level images extracted from the Land Use/Cover Area Frame Survey (LUCAS) dataset provided by Eurostat. This dataset is designed for semantic segmentation tasks, focusing on distinguishing between different land cover categories, including agricultural and natural landscapes. The dataset covers the survey year 2018 and includes north looking images with both full masks and partial masks, where certain areas are not delineated.

land-use and land-cover change
segmentation
biodiversity
ml
monitoring
vector
image
street level imagery
landscape elements
computer vision


Repository of datasets useful for testing spatial queries of GeoParquet datasets

Useful to demonstrate whether predicate pushdown is supported on top-level fields or nested group members for example.


Auth Test

test


test repository

No Description Provided


Test Repository

No Description Provided


Fields of The World (FTW) - Archive

This is an archive version of the main Fields of The World (FTW) benchmarking dataset. Currently, it's mainly for internal use.


Fields of The World: Field Boundaries for Austria

Austrian field boundaries and land parcels from the Agrarmarkt Austria paying agency, containing reference geometries and crop classifications for 2021 under EU agricultural regulations.

agriculture field-boundaries land-parcels crop-classification austria reference-parcels

## Description

The dataset comprises all reference parcels established by the Paying Agency Agrarmarkt Austria and covered landscape elements (landscape element layers) within the meaning of Article 5 of Regulation (EU) No 640/2014 and Regulation of the competent Federal Ministry laying down horizontal rules for the area of the common agricultural policy (Horizontal GAP Regulation) StF: BGBl.

- **Source Data Curator:** [Euro Crops](https://github.com/maja601/EuroCrops/wiki/Austria)
- **Source Data Provider:** [Publications Office of the European Union.](https://data.europa.eu/data/datasets/ama_invekosreferenzensterreich2021?locale=en)
- **Converted by:** Kerner Lab (ASU)
- **License:** CC-BY-4.0
- **Projection:** EPSG:4326
- **Dataset Validity:** 2021

---

## Attributes

| Property                  | Data Type |
| ------------------------- | --------- |
| id                        | string    |
| determination_datetime    | datetime  |
| area                      | float     |
| geometry                  | polygon   |
| crop_id                   | int64     |
| crop_name                 | string    |

Source Cooperative product page

README

Dataset files (1 × .parquet)


Fields of The World: Field Boundaries for Belgium

Field boundaries and crop classifications for Belgium from the 2021 agricultural campaign, sourced from the Agriculture and Sea Fisheries Agency's annual crop inventory. The dataset documents field geometries, crop types, and previous crop information with a reference date of April 30, 2021.

agriculture field-boundaries crop-classification farm-boundaries

## Description

The inventory of the original plots is done annually by the Agriculture and Sea Fisheries Agency in the context of the payment of the (co-financed) European agricultural subsidies and the Flemish manure legislation.

As standard, publishers exchanged plots used for the main crop on the final submission date of the collective application (reference date). For the 2021 campaign this is 04/30/2021.The metadata also provides the date that is being prepared for the 2021 campaign. This is the final condition of the plots on 15/03/2022. The dataset has an informative value for a generalized use and is in no way intended for control purposes or individualized use.

_The field boundaries are valid for the year 2021_

- **Source Data Curator:** [EuroCrops](https://github.com/maja601/EuroCrops/wiki/Belgium)
- **Original Data Provider:** [The Agriculture and Sea Fisheries Agency](https://metadata.vlaanderen.be/srv/dut/catalog.search#/metadata/d869db4a-73f4-470e-b5ca-a0f7cd3ab585)
- **Converted by:** Kerner Lab (ASU)
- **License:** No restrictions on public access
- **Projection:** EPSG:4326

---

## Columns

| Property               | Data Type |
| ---------------------- | --------- |
| id                     | string    |
| determination_datetime | datetime  |
| area                   | float     |
| pre_crop_code          | string    |
| pre_crop_name          | string    |
| crop_code              | string    |
| crop_name              | string    |
| second_post_crop_name  | string    |
| EC_trans_n             | string    |
| EC_hcat_n              | string    |

Source Cooperative product page

README

Dataset files (1 × .parquet)

No restrictions on public access; original source is Belgian government data


Fields of The World: Field Boundaries for Brazil

Monthly land use classification data for 1,854 agricultural fields in western Bahia, Brazil (October 2019–September 2020). The dataset documents primary crop types and land use patterns in the Luís Eduardo Magalhães region derived from remote sensing analysis.

agriculture field-boundaries land-use remote-sensing crop-classification

## Description

This dataset is the supplementary data of a paper published in the Data in Brief Journal. The dataset, in ESRI shapefile format (spatial reference system: WGS 84, EPSG: 4326), provides monthly land use information about 1854 fields from October 2019 to September 2020 from Luís Eduardo Magalhães (LEM) and other municipalities in the west of Bahia state, Brazil. The majority of the 16 land uses classes are related to crops. The paper describes the applied data collection methods and procedures.

`Oldoni, Lucas Volochen ; Sanches, Ieda Del’Arco ; Picoli, Michelle Cristina A. ; Covre, Renan Moreira ; Fronza, José Guilherme  (2020), “LEM+ dataset: for agricultural remote sensing applications ”, Mendeley Data, V1, doi: 10.17632/vz6d7tw87f.1.
`

- **Source Data Curator:** [Lucas Volochen Oldoni, Ieda Del’Arco Sanches, Michelle Cristina A. Picoli, Renan Moreira Covre, José Guilherme Fronza](https://data.mendeley.com/datasets/vz6d7tw87f/1)
- **Source Data Provider:** [Lucas Volochen Oldoni, Ieda Del’Arco Sanches, Michelle Cristina A. Picoli, Renan Moreira Covre, José Guilherme Fronza](https://data.mendeley.com/datasets/vz6d7tw87f/1)
- **Converted by:** Kerner Lab (ASU)
- **License:** CC-BY-4.0
- **Projection:** EPSG:4326
- **Dataset Validity:** 2020

---

## Attributes

| Property                  | Data Type |
| ------------------------- | --------- |
| id                        | string    |
| determination_datetime    | datetime  |
| geometry                  | polygon   |
| crop_id                   | int64     |
| crop_name                 | string    |

Oldoni, Sanches, Picoli, Covre, and Fronza

Source Cooperative product page

README

Dataset files (1 × .parquet)

Oldoni, Lucas Volochen; Sanches, Ieda Del'Arco; Picoli, Michelle Cristina A.; Covre, Renan Moreira; Fronza, José Guilherme (2020). LEM+ dataset: for agricultural remote sensing applications. Mendeley Data, V1. doi: 10.17632/vz6d7tw87f.1


Fields of The World: Field Boundaries for Cambodia

Dataset of 318,088 field polygons across Cambodia derived from the AI4SmallFarms project, digitized from satellite imagery with rigorous quality control. Valid as of August 2021.

agriculture field-boundaries crop-classification remote-sensing smallholder-farming

## Description

This dataset is a obtained from AI4SmallFarms dataset publised by the University of Twente Research Information. It is updated by the requiremnt of the fieldscapes project.

The large-scale data set consists of 318,088 field polygons divided into 32 tiles of approximately 5×5 km distributed across Cambodia covering a range of fields and diverse landscape types. The field polygons have been meticulously digitized by original author from satellite images, following a rigorous multi-step quality control process and topological consistency checks.

_The field boundaries are valid for the August 2021_

- **Original Data Provider:** [DATA Archiving and Networked Services (DANS)](https://research.tudelft.nl/en/publications/ai4smallfarms-a-dataset-for-crop-field-delineation-in-southeast-a)
- **Converted by:** Kerner Lab (ASU)
- **License:** CC-BY-4.0
- **Projection:** EPSG:4326

---

## Columns

| Property               | Data Type |
| ---------------------- | --------- |
| id                     | string    |
| determination_datetime | datetime  |
| \_predicate            | string    |

Source Cooperative product page

README

Dataset files (1 × .parquet)


Fields of The World: Field Boundaries for Corsica

This dataset is an anonymized subset of France's RPG (Republicateur Parcelles Graphiques), a geographical reference database used for Common Agricultural Policy aid administration. It contains vector field boundaries and crop classifications for agricultural plots in Corsica.

agriculture field-boundaries crop-classification land-use land-parcels

## Description

This dataset is a subset of the dataset obtained from the RPG Website, RPG is a geographical database used as a reference for the processing of aid from the common agricultural policy (CAP). The anonymized version distributed here as part of the public service for making reference data available contains graphic data for plots (basic land unit for farmers' declaration) with their main crop.

- **Source Data Curator:** [Republique Francaise - Geoservices](https://geoservices.ign.fr/rpg)
- **Source Data Provider:** [Republique Francaise - Geoservices](https://geoservices.ign.fr/rpg)
- **Converted by:** Kerner Lab (ASU)
- **License:** CC-BY-2.0
- **Projection:** EPSG:4326
- **Dataset Validity:** 2022

---

## Attributes

| Property                  | Data Type |
| ------------------------- | --------- |
| id                        | string    |
| determination_datetime    | datetime  |
| area                      | float     |
| geometry                  | polygon   |
| crop_id                   | int64     |
| crop_name                 | string    |

Source Cooperative product page

README

Dataset files (1 × .parquet)


Fields of The World: Field Boundaries for Croatia

Field boundaries representing agricultural land parcels in Croatia, vectorized from orthophoto imagery and classified by land use type. Data originates from the ARKOD registry maintained by the Croatian Agency for Payments, with field boundaries valid for 2023.

agriculture field-boundaries land-use agricultural-parcels crop-classification

## Description

This dataset is adapted from the data published by Republic of Croatia, in the ARKOD system. ARKOD is a record of the use of agricultural land in the territory of the Republic of Croatia, which is maintained in digital graphic form by the Agency for Payments.

ARKOD plots represent uninterrupted areas of agricultural land cultivated by one farmer, classified according to the type of use of agricultural land from gievn Annex I of the Ordinance on records of use of agricultural land.

Process for registration of agricultural land - vectorization of ARKOD polygons is carried out by the method of photointerpretation of agricultural land on DOF bases on a scale of 1:5000 in branches of the Agency for Payments, statements of farmers on the use of agricultural land with proof of ownership or possession.

_The field boundaries are valid for the year 2023_

- **Original Data Provider:** [The Agency for Payments in Agriculture, Fisheries and Rural Development](https://www.apprrr.hr/prostorni-podaci-servisi/)
- **Converted by:** Kerner Lab (ASU)
- **License:** Open Data
- **Projection:** EPSG:4326

---

## Columns

| Property                         | Data Type |
| -------------------------------- | --------- |
| id                               | string    |
| determination_datetime           | datetime  |
| area                             | float     |
| land_use_id                      | string    |
| slope                            | float     |
| average_height                   | float     |
| eligibility_coef                 | float     |
| mines_status                     | string    |
| water_protect_zone               | string    |
| restricted_surface               | string    |
| restricted_surface_area          | float     |
| vulnerable_areas                 | float     |
| high_degree_natural_value_meadow | string    |
| irrigation                       | int32     |
| irrigation_source                | float     |
| irrigation_type                  | float     |

Source Cooperative product page

README

Dataset files (1 × .parquet)

Designated as 'Open Data' by the Republic of Croatia; specific SPDX license not specified


Fields of The World: Field Boundaries for Denmark

The Danish Land Parcel Identification System (LPIS) maps agricultural land through field blocks—geographically coherent units bounded by permanent landscape features. Covering approximately 2.8 million hectares with continuously updated boundaries, each field block record includes area, eligibility, and support classification attributes.

agriculture field-blocks lpis land-parcels field-boundaries crop-cultivation

## Description

Danish Land Parcel Identification System (LPIS) data collection or (field blocks/ 'Markblokke') is managed by The Danish Agency for Agriculture.

The field block map is a digital field map, with agricultural areas gathered in field blocks. A field block is a geographically coherent unit consisting of agricultural land. The boundaries of the field blocks typically follow permanent divisions in the landscape. The map is used in the administration of cases linked to the geographical determination of cultivation areas, primarily by the EU's area-based support schemes. The field block map is continuously updated on the basis of orthophotos and reports from farmers and authorities. The field block map covers approx. 2.8 million hectares of agricultural land. Each field block is identified with a field block number, area, type. A field block means a continuous area on which one or more farmers grow one or more different crops. The map is updated every night with changes from case processing.

_The field boundaries are valid for the year 2021_

- **Original Data Provider:** [The Danish Agency for Agriculture](https://geodata-info.dk/srv/eng/catalog.search#/metadata/d91b2c99-d9b0-4e6d-b323-20ac80548186)
- **Converted by:** Kerner Lab (ASU)
- **License:** CC0-1.0
- **Projection:** EPSG:4326

---

## Columns

| Property               | Data Type |
| ---------------------- | --------- |
| id                     | string    |
| determination_datetime | datetime  |
| area                   | float     |
| eligible_area          | float     |
| ineligible_area        | float     |
| deduction              | float     |
| support_proc           | float     |
| block_type             | string    |

Source Cooperative product page

README

Dataset files (1 × .parquet)


Fields of The World: Field Boundaries for Estonia

Estonian agricultural field boundaries for 2021 from the INSPIRE GSAA (Geospatial Aid Application), originally provided by The Agricultural Registers and Information Agency (PRIA). The dataset includes field-level attributes such as land use classification, requested culture, and crop detection status.

agriculture field-boundaries gsaa iacs crop-monitoring land-use

## Description

Geospatial Aid Application Estonia Agricultural parcels. The original dataset in obtained from the INSPIRE theme GSAA ( specifically Geospatial Aid Application Estonia Agricultural parcels) and download service provided by PRIA, through which the data layer Fields and Ecological Areas (GSAA) is made available. The data comes from the PRIA database.

This dataset is later on adapted by Eurocrops then its subset consumed by the fieldscapes project.

_The field boundaries are valid for the year 2021_

- **Source Data Provider:** [EuroCrop](https://zenodo.org/records/8229128/files/EE_2021.zip?download=1)
- **Original Data Provider:** [The Agricultural Registers and Information Agency](https://inspire-geoportal.ec.europa.eu/srv/eng/catalog.search#/results?country=ee&view=IACSOverview&theme=none&iacs=gsaa&legislation=none&envdomain=none&basedomain=0)
- **Converted by:** Kerner Lab (ASU)
- **License:** CC-3.0
- **Projection:** EPSG:4326

---

## Columns

| Property                | Data Type |
| ----------------------- | --------- |
| id                      | string    |
| determination_datetime  | datetime  |
| area                    | float     |
| requested_culture       | string    |
| requested_land_use      | string    |
| mowing_detection_status | string    |
| mowing_detection_period | string    |
| requested_support       | string    |
| EC_trans_n              | string    |
| EC_hcat_n               | string    |

Source Cooperative product page

README

Dataset files (1 × .parquet)

Source notation 'CC-3.0' interpreted as CC-BY-3.0 (standard SPDX format)


Fields of The World: Field Boundaries for Finland

Finnish agricultural land use dataset covering arable land, fallow land, permanent grasslands, natural pastures, and permanent crops for the year 2021. Published by the Finnish Food Authority under the INSPIRE Directive, subset of field boundary classifications.

agriculture field-boundaries land-use arable-land pastures permanent-crops

## Description

The Finnish Food Authority has published spatial data sets under the INSPIRE Directive since 2020. Since 2023, the published data has contained high-value data sets.

Arable land is land cultivated for plant production or fallow land available for cultivation Fallen land is arable agricultural land that is not used for agricultural production. Fallow land can be either field or permanent grass. Fallow land must be kept open so that it is suitable for grazing or cultivation without having to prepare it with anything other than conventional agricultural methods and machinery. Take care to keep the area open if necessary, e.g. by mowing or grazing.

Permanent turf is a block that is used to grow hay and grass plants that have been in the same place for more than 5 years, either by self-renewal or by sowing. The land use type of permanent grass can be a field or natural pasture and meadow. A maximum of 50 trees per hectare are allowed to grow scattered on permanent lawns. Tree-like plants over two meters tall, with one trunk or several trunks joined together at the base, are considered trees. Trees are also plants with woody stems that are 0.5–2 meters tall and are not suitable for feeding farm animals. Leafy shrubs and saplings of deciduous trees suitable for feeding farm animals may grow in the area if they are less than 50 percent of the area of the eligible growth block. The share of hay and grass fodder plants must be more than 50 percent of the eligible area of the growth block, even in those blocks of permanent grass that have either trees or leafy bushes or both.

Natural pastures are areas of permanent grass, the land use type of which is natural pasture and meadow.

A permanent plant means a block of land use type, which has plants that are not part of the crop rotation, which are grown for at least 5 years and from which a repeated harvest is obtained. These plants include, for example, fruit trees, berry bushes and ornamental plants.

_The field boundaries are valid for the year 2021_

- **Original Data Provider:** [The Food Agency of Finland](https://www.paikkatietohakemisto.fi/geonetwork/srv/eng/catalog.search#/metadata/d85fbeba-fe58-4244-83d4-6f6dd8e4208d)
- **Converted by:** Kerner Lab (ASU)
- **License:** CC-BY-4.0
- **Projection:** EPSG:4326

---

## Columns

| Property               | Data Type |
| ---------------------- | --------- |
| id                     | string    |
| determination_datetime | datetime  |
| land_use               | string    |

Source Cooperative product page

README

Dataset files (1 × .parquet)


Fields of The World: Field Boundaries for France

Anonymized agricultural field plot boundaries and crop classifications from France's graphic parcel register (RPG), an official Common Agricultural Policy (CAP) reference database. Contains plot-level data managed by the Services and Payments Agency (ASP) since 2007, with field boundaries valid for 2020.

field-boundaries crop-classification agriculture land-use land-parcels lpis common-agricultural-policy

## Description

This dataset is a subset of graphic parcel register dataset managed by the ASP (Service and Payment Agency). The graphic parcel register is a geographical database used as a reference for the processing of aid from the common agricultural policy (CAP).

The anonymized version is distributed as part of the public service for making reference data available contains graphic data for plots (basic land unit for farmers' declaration) with their main crop. This data has been produced by the Services and Payment Agency (ASP) since 2007.

An anonymous version of the graphic data of the RPG associated with some of the data declared by the operators is disseminated. This data provides detailed information on land use and land structures. For the 2007 to 2014 versions of the RPG, the data disseminated only includes blocks. From the 2015 edition of the RPG, the data disseminated includes blocks and agricultural plots: information on land use and type of crop goes back down to the plot level, information on land structures disappears. From the 2022 edition of the RPG, the data disseminated only includes agricultural plots. The ASP now entrusts the IGN with the dissemination of anonymized data from the RPG.

The anonymous RPG data are vintage and contain plots corresponding to those declared for campaign N in their known situation and decided by the administration, generally on January 1 of year N+1. These data cover the entire French territory, including Mayotte and Saint-Martin, but excluding Saint-Barthélemy.

_The field boundaries are valid for the year 2020_

- **Original Data Provider:** [The Services and Payments Agency(ASP)](https://geoservices.ign.fr/rpg#telechargementrpg2021)
- **Converted by:** Kerner Lab (ASU)
- **License:** Open Licence
- **Projection:** EPSG:4326

---

## Columns

| Property               | Data Type |
| ---------------------- | --------- |
| id                     | string    |
| determination_datetime | datetime  |
| area                   | float     |
| crop_code              | string    |
| crop_code_i            | uint8     |

Source Cooperative product page

README

Dataset files (1 × .parquet)

French Licence Ouverte (Open Licence), an open government data license.