Flight Tracks

AeroWhere Flight Tracks is a database of historic aircraft flight track segments over the contiguous United States (CONUS), derived from ADS-B (Automatic Dependent Surveillance–Broadcast) position reports captured by ground receiver networks. Each record is a GIS linestring geometry for a single flight track segment, attributed with ICAO hex ID, aircraft registration, type, UTC timestamps, and height above ground level (AGL) computed using a USGS 3DEP digital elevation model. Data are stored as hive-partitioned GeoParquet files indexed by zoom-level-8 quadkey, year, month, and day.

geoparquet time-series digital-elevation-model aviation flight-tracking ads-b trajectories

flights-2023-09-26.parquet

[geometry] 2D GeoArrow linestring representing a single flight track segment in EPSG:4326

[spatial_index] Zoom-level-8 quadkey (EPSG:3857 WebMercatorQuad) used for hive partitioning

flights-2024-10-04.parquet

flights-2024-11-11.parquet

flights-2025-06-05.parquet

flights-2024-12-23.parquet

flights-2024-08-10.parquet

flights-2023-11-20.parquet

flights-2024-01-22.parquet

flights-2025-08-08.parquet

flights-2024-01-13.parquet

flights-2024-12-09.parquet

flights-2023-12-27.parquet

flights-2024-01-17.parquet

flights-2023-07-16.parquet

flights-2023-07-03.parquet

flights-2023-04-12.parquet

flights-2025-03-29.parquet

flights-2023-10-25.parquet

flights-2025-10-07.parquet

flights-2025-04-19.parquet

flights-2023-08-24.parquet

flights-2024-10-01.parquet

flights-2024-10-30.parquet

flights-2025-04-28.parquet

flights-2024-07-31.parquet

flights-2024-08-15.parquet

flights-2024-09-18.parquet

flights-2025-03-21.parquet

flights-2025-09-14.parquet

flights-2026-01-02.parquet

flights-2023-04-06.parquet

Source Cooperative product page

## README.md
# AeroWhere Flight Tracks
## Description
The AeroWhere Flight Tracks database captures historic aircraft flight tracks as GIS linestring geometries, attributed with height above ground level (AGL) and other identifying information about the flight (timestamp, ICAO, aircraft type, etc.). The currently available data is limited to the contiguous United States (CONUS) and includes flights spanning the time period noted in the accompanying [Date Range](https://data.source.coop/root-geospatial/flight-tracks/DATE-RANGE.md) file.

The flight tracks are derived from Automatic Dependent Surveillance–Broadcast (ADS-B) position traces, captured by a network of ground receivers, collected and made available via [ADS-B LOL historical data archive](https://www.adsb.lol/docs/open-data/historical/) (see [Source Data](#source-data) for more information).

The source ADS-B files have been parsed into GIS format by connecting ADS-B positions of each unique aircraft in chronological order. Gaps between consecutive locations ≥ 30 km were not connected to limit uncertainty about aircraft positioning in spans of this size. Altitudes were converted to height above ground through the use of a digital elevation model (DEM) (see [Source Data](#source-data)). Only a select set of attributes from the source ADS-B files are currently included in the flight tracks dataset (see [Schema](#schema)).

[![AeroWhere Flight Tracks Screenshot](https://data.source.coop/root-geospatial/flight-tracks/docs/banner.png "AeroWhere Flight Tracks Viewer")](https://aerowhere.com/flight-tracks?lat=40.7178&lng=-74.0952&zoom=10.00&date=2023-11-27)

## Data Format
### GeoParquet
The data are provided as a hive-partitioned GeoParquet dataset. The GeoParquet files follow the specification for GeoParquet v1.1.0 and geometries are encoded as 2-D linestrings in GeoArrow format.

Each geometry represents a single segment of a flight track. Geometries are encoded in the direction of travel of the flight track (i.e., first vertex is the start of the segment, last vertex is the end of the segment). All geometries are in the EPSG:4326 coordinate reference system.

### Partitioning
The dataset uses hive partitioning to split the dataset into files of reasonable size to enable faster acess. The partitioning scheme is intended to be spatiotemporal, first splitting by year, month, and day. The structure is:

`quadkey_z8={quadkey}/year={year}/month={month}/day={day}/flights-{year}-{MONTH}-{DAY}.parquet`

where:
- `quadkey` is the zoom level 8 tile quadkey in the spherical mercator EPSG:3857 grid
- `year` is the 4-digit year of the partition (e.g., `2025`)
- `month` is the month of the partition *without* zero padding (i.e., `1`, `2`, ..., `12`)
- `MONTH` is the month of the partition *with* zero padding (i.e., `01`, `02`, ..., `12`)
- `day` is the day of the partition  *without* zero padding (i.e., `1`, `2`, ..., `31`)
- `DAY` is the day of the partition  *with* zero padding (i.e., `01`, `02`, ..., `31`)

> **Important** \
> The tile based partitioning is based on spatial intersection between flight tracks and the tiles. As a result, a single flight track may occur across multiple partitions if it overlaps multiple tiles. For certain uses cases, this means you may need to deduplicate records. This can be done using the `end_t` (or `start_t`) and `icao` fields as a compound identifier for unique flight segments (see [Data Access Examples](#data-access-examples-python)). Note that this also means that the data for a single partition will not strictly adhere to the tile bounds (i.e., the data won't be clipped exactly to the tile edges).

### Schema
Each partition uses the following schema:
- `geometry` (list[struct(x, y)]): 2D linestring geometry representing a single segment of a flight track.
- `icao` (string): Hex ID of the aircraft
- `r` (string): Aircraft registration
- `aircraft_type` (string): Aircraft type
- `desc` (string): Aircraft type description
- `start_t` (timestamp[ns]): Timestamp (in UTC) for the start (first vertex) of the segment.
- `end_t` (timestamp[ns, tz=UTC]): Timestamp (in UTC) for the end (last vertex) of the segment.
- `start_height_agl_ft` (int32): Height above ground level in feet for the start (first vertex) of the segment. Derived from altitude reported in ADS-B and ground elevation from DEM.
- `end_height_agl_ft` (int32): Height above ground level in feet for the end (last vertex) of the segment. Derived from altitude reported in ADS-B and ground elevation from DEM.

## Data Access Examples (Python)
Required dependencies:
- geopandas: https://geopandas.org/en/stable/
- pyarrow: https://arrow.apache.org/docs/python/index.html
- morecantile: https://developmentseed.org/morecantile/

### Access partitions with a loop
```python
import geopandas  as gpd
import pandas as pd

quadkeys = ["02310101", "02310103"]
year = 2023
month = 11
start_day = 15
end_day = 16

base_url = "s3://us-west-2.opendata.source.coop/root-geospatial/flight-tracks"
part_dfs = []
for quadkey in quadkeys:
    for day in range(start_day, end_day + 1):
        full_url = (
            f"{base_url}/quadkey_z8={quadkey}/year={year}/month={month}/day={day}/"
        )
        part_df = gpd.read_parquet(full_url)
        part_dfs.append(part_df)

df = pd.concat(part_dfs, ignore_index=True)

# remove duplicates where lines overlap multiple tiles
df.drop_duplicates(subset=["icao", "end_t"], inplace=True)

```

### Access partitions with hive filters (slower but more flexible)
```python
import geopandas  as gpd
import pyarrow as pa
import pyarrow.dataset as ds

# Set schema to ensure quadkeys are treated as string, not int
schema = pa.schema([
    ("quadkey_z8", pa.string()),
    ("year", pa.int32()),
    ("month", pa.int8()),
    ("day", pa.int8())
])

quadkeys = ["02310101", "02310103"]
year = 2023
month = 11
start_day = 15
end_day = 16

df = gpd.read_parquet(
    "s3://us-west-2.opendata.source.coop/root-geospatial/flight-tracks/",
    filters=[
        ("quadkey_z8", "in", quadkeys),
        ("year", "==", year),
        ("month", "==", month),
        ("day", ">=", start_day),
        ("day", "<=", end_day)],
    partitioning=ds.HivePartitioning(schema=schema)
)

# remove duplicates where lines overlap multiple tiles
df.drop_duplicates(subset=["icao", "end_t"], inplace=True)

```

### Finding quadkeys covering a bounding box
```python
import morecantile

# bbox in format: xmin, ymin, xmax, ymax
bbox = [-104.7943, 39.7867, -104.3485, 40.0569]

tms = morecantile.tms.get("WebMercatorQuad")
tiles_gen = tms.tiles(*bbox, zooms=[8])
quadkeys = [tms.quadkey(t) for t in tiles_gen]
print(quadkeys)

```

## Limitations
The currently available data is limited to the contiguous United States (CONUS) and includes flights spanning the time period noted in the accompanying [Date Range](https://data.source.coop/root-geospatial/flight-tracks/DATE-RANGE.md) file.

Because the dataset is based on a network of ADS-B ground receivers, flight tracks may be incomplete or missing for any given location and date. Complete coverage of all real-world flights is not guaranteed.

## Citation and Attribution
If you are the author of a publication or research report that uses these data, please include the following citation:

Root Geospatial LLC. 2025. "AeroWhere Flight Tracks: Historical Flight Tracks Dataset". Source Cooperative, https://source.coop/root-geospatial/flight-tracks. Accesed [ACCESS DATE].

For use in web maps, please include the following attribution:
[© AeroWhere by Root Geospatial](https://rootgeo.com/).

## Data Viewer and Exports
A data viewer with functionality for exporting small subset of the data is available at https://aerowhere.com/flight-tracks.

## Source Data
Flight tracks positions and attributes are derived from ADS-B.lol Historical Data: https://www.adsb.lol/docs/open-data/historical/.

Attributes indicating height AGL values are derived from a combination of the altitude values provided in the ADS-B.LOL files and a USGS 3DEP program 1 arc-second DEM (https://www.usgs.gov/3d-elevation-program/about-3dep-products-services).

## Licensing
This data is made available under the Open Database License (ODbL) v1.0.
The full license text is included in the file [LICENSE.txt](https://data.source.coop/root-geospatial/flight-tracks/LICENSE.txt) and can also be found at http://opendatacommons.org/licenses/odbl/1.0/.

extent.spatial.bbox: omitted; spatial reference is categorical via [flights-2023-09-26.geometry (geometry), flights-2023-09-26.quadkey_z8 (spatial_index), flights-2024-10-04.geometry (geometry), flights-2024-10-04.quadkey_z8 (spatial_index), flights-2024-11-11.geometry (geometry), flights-2024-11-11.quadkey_z8 (spatial_index), flights-2025-06-05.geometry (geometry), flights-2025-06-05.quadkey_z8 (spatial_index), flights-2024-12-23.geometry (geometry), flights-2024-12-23.quadkey_z8 (spatial_index), flights-2024-08-10.geometry (geometry), flights-2024-08-10.quadkey_z8 (spatial_index), flights-2023-11-20.geometry (geometry), flights-2023-11-20.quadkey_z8 (spatial_index), flights-2024-01-22.geometry (geometry), flights-2024-01-22.quadkey_z8 (spatial_index), flights-2025-08-08.geometry (geometry), flights-2025-08-08.quadkey_z8 (spatial_index), flights-2024-01-13.geometry (geometry), flights-2024-01-13.quadkey_z8 (spatial_index), flights-2024-12-09.geometry (geometry), flights-2024-12-09.quadkey_z8 (spatial_index), flights-2023-12-27.geometry (geometry), flights-2023-12-27.quadkey_z8 (spatial_index), flights-2024-01-17.geometry (geometry), flights-2024-01-17.quadkey_z8 (spatial_index), flights-2023-07-16.geometry (geometry), flights-2023-07-16.quadkey_z8 (spatial_index), flights-2023-07-03.geometry (geometry), flights-2023-07-03.quadkey_z8 (spatial_index), flights-2023-04-12.geometry (geometry), flights-2023-04-12.quadkey_z8 (spatial_index), flights-2025-03-29.geometry (geometry), flights-2025-03-29.quadkey_z8 (spatial_index), flights-2023-10-25.geometry (geometry), flights-2023-10-25.quadkey_z8 (spatial_index), flights-2025-10-07.geometry (geometry), flights-2025-10-07.quadkey_z8 (spatial_index), flights-2025-04-19.geometry (geometry), flights-2025-04-19.quadkey_z8 (spatial_index), flights-2023-08-24.geometry (geometry), flights-2023-08-24.quadkey_z8 (spatial_index), flights-2024-10-01.geometry (geometry), flights-2024-10-01.quadkey_z8 (spatial_index), flights-2024-10-30.geometry (geometry), flights-2024-10-30.quadkey_z8 (spatial_index), flights-2025-04-28.geometry (geometry), flights-2025-04-28.quadkey_z8 (spatial_index), flights-2024-07-31.geometry (geometry), flights-2024-07-31.quadkey_z8 (spatial_index), flights-2024-08-15.geometry (geometry), flights-2024-08-15.quadkey_z8 (spatial_index), flights-2024-09-18.geometry (geometry), flights-2024-09-18.quadkey_z8 (spatial_index), flights-2025-03-21.geometry (geometry), flights-2025-03-21.quadkey_z8 (spatial_index), flights-2025-09-14.geometry (geometry), flights-2025-09-14.quadkey_z8 (spatial_index), flights-2026-01-02.geometry (geometry), flights-2026-01-02.quadkey_z8 (spatial_index), flights-2023-04-06.geometry (geometry), flights-2023-04-06.quadkey_z8 (spatial_index)]. Add bbox once the placename → bbox gazetteer covers it.


long_lfmc_test

Test upload for our long lfmc dataset.


sentinel1_lfmc

Sentinel-1 based Live Fuel Moisture Content dataset. This is implemented via the method described in Rao et al., 2020, Remote Sensing of Environment: "SAR-enhanced mapping of live fuel moisture content"


Demo for ESIP 2025 Summer Meeting

Demo for ESIP 2025 Summer Meeting


LiveOcean

LiveOcean ROMS Ocean Forecast for Puget Sound


NCEI Estuarine Bathymetry

Gridded bathymetric Digital Elevation Models for 70 estuaries in the conterminous United States, produced by the NOAA National Ocean Service from 150 years of hydrographic survey data. Available as 30-meter resolution NetCDF files with accompanying XML metadata, covering major U.S. estuaries with sufficient historical survey data for detailed bathymetric processing.

bathymetry oceanography topography digital-elevation-model estuaries hydrography hydrology coastal netcdf

NOAA National Centers for Environmental Information (NCEI)

Dataset files (71 x .nc)

Source Cooperative product page

## README.md
# NOAA Estuarine Bathymetry DEMs
The National Ocean Service (NOS) Estuarine Bathymetric Digital Elevation Models are gridded bathymetry datasets interpolated from 150 years’ worth of hydrographic survey data collected by the former NOS Special Projects Office. The initiative produced datasets for 70 estuaries in the conterminous United States with sufficient data coverage to support detailed bathymetric processing.

Estuarine bathymetric data are available as gridded products with 30 meter spatial resolution as NetCDF files, along with XML metadata files.

## Access Data
### Stream Data
Here's an example of how to explore the data interactively in a Jupyter Notebook using Python:
``` python
import fsspec
import xarray as xr
import hvplot.xarray

fs = fsspec.filesystem('s3', anon=True, endpoint_url='https://data.source.coop')
fs.ls('rsignell/ncei-estuarine-bathymetry')

ds = xr.open_dataset(fs.open('rsignell/ncei-estuarine-bathymetry/Buzzards_Bay/buzzards_bay_m010_30m.nc'))
ds['Band1'].hvplot(x='x', y='y', crs='EPSG:26919', rasterize=True, tiles='OSM', cmap='viridis')
```

license: AI returned 'unknown' (README does not state a license). Verify with publisher before flipping to _state: reviewed.


Sentinel2 Crop labels from ground surveys for six locations in Rwanda

These crop labels were developed from ground surveys at six specific locations in Rwanda in winter (season A) 2021-2022.  The purpose was to compare this method of crop label development against using high resolution drone imagery.  Six major crops were identified: maize, banana, cassava, beans, sweet potatoes, and Irish potatoes.


Sentinel2 crop labels from drone imagery for six locations in Rwanda

These crop labels were developed from drone imagery at six specific locations in Rwanda in winter (season A) 2021-2022.  The purpose was to compare this method of crop label development against using field surveys.  Six major crops were identified: maize, banana, cassava, beans, sweet potatoes, and Irish potatoes.


Rwanda Crop and Land Cover Labels Using Drone Imagery and Field Observations

RTI International (RTI) generated a detailed dataset offering precise crop and land cover classifications across six specific sites in Rwanda. Collected through field observations and high-resolution drone imagery, encompassing 5763 labels.


Overture Maps - Places Dataset - Riyadh, Saudi Arabia

A geospatial dataset of places and areas of interest (AOIs) in Riyadh, Saudi Arabia, containing 32,039 location records derived from Overture Maps in GeoParquet format.

Source Cooperative product page

Dataset files (1 × .parquet)

Overture Maps data terms; consult source.coop for licensing details


test-1


A collection of ecological model outputs — primarily species distribution models — for Antarctica and the Southern Ocean, compiled by the SCAR Expert Group on Biodiversity Informatics. Outputs are provided as cloud-optimized GeoTIFFs (one file per species, with multiple bands where uncertainty estimates or multiple output variables are included), each retained in its original coordinate reference system and spatial resolution. The collection covers circumpolar and regional domains across the Antarctic continent and Southern Ocean.

southern-ocean species-distribution-model remote-sensing biodiversity ecology marine oceanography

Dataset files (418 x .tif)

Source Cooperative product page

## README.md
# DistAnt

## Overview

This repository provides access to a collection
of ecological model outputs (species distribution and similar models)
from Antarctica and the Southern Ocean.
It is a project of the SCAR [Expert Group on
Biodiversity Informatics](https://scar.org/science/egabi/home/) in
conjunction with
[ADVANCE](https://www.belspo.be/belspo/impuls/project_en.stm#ADVANCE)
(Royal Belgian Institute of Natural Sciences) and the Integrated Digital
East Antarctica program at the [Australian Antarctic
Division](https://www.antarctica.gov.au/science/).

**Please note:** the inclusion of a layer in this collection is not an
endorsement of its quality or suitability for your intended purpose.
Users should consult the associated publication for details on the
source data and modelling processes. We encourage users to contact the
original publication authors to discuss their intended use of these
layers.

Model outputs are provided as cloud-optimized geotiffs (COGs), with
generally one file per species. The COG will have multiple bands if the
original model predictions include uncertainty estimates or multiple
model output variables. Each output has been kept on its original
coordinate reference system (map projection) and spatial resolution.

### Citing

The layers in this collection have been re-released under their original
license where applicable, or a CC-BY licence otherwise. Please cite when
using, and also cite the original data sources used. For example:

> Kovacs J (2020) A model of my favourite Southern Ocean species. *Journal of Southern Ocean Stuff* **123**:1-10. Data obtained from the SCAR DistAnt Ecological Model Output Repository, 10.5281/zenodo.10910075

Individual data sources might have varying licence conditions: consult
the `licence` field for details.

## Contribute data

If you have generated some model outputs that would be suitable for this
collection, or are aware of some that we haven’t yet included, please
[open an issue and let us
know](https://github.com/SCAR/distant/issues/new/choose).

The focus for this repository --- at least initially --- is on circumpolar
layers that encompass the whole Southern Ocean or whole Antarctic
continent. However, regional outputs that cover smaller domains will be
considered, particularly if they relate to a taxon or output variable for
which there is no circumpolar output.

## Project contributors

  - Charlie Plasman, Anton Van de Putte (Royal Belgian Institute of
    Natural Sciences, Université Libre de Bruxelles)
  - Salomé Fabri-Ruiz (Institut Français de Recherche pour
    l’Exploitation de la Mer)
  - Jennifer Freer, Huw Griffiths (British Antarctic Survey)
  - David Green, Benjamin Viola, Briannyn Woods (Institute for Marine
    and Antarctic Studies)
  - Jean-Olivier Irisson (Institut de la Mer de Villefranche)
  - Lucas Krüger (Instituto Antártico Chileno)
  - Benjamin Merkel (Norwegian Polar Institute)
  - Matt Pinkerton (National Institute of Water and Atmospheric
    Research)
  - Ryan Reisinger (University of Southampton)
  - Giovanni Testa, Andrea Piñones (Centro de Investigación en Dinámica
    de Ecosistemas Marinos de Altas Latitudes)
  - Anikó Tóth (University of New South Wales)
  - José Xavier (University of Coimbra, British Antarctic Survey)
  - Ben Raymond, Aleks Terauds (Australian Antarctic Division)

## Contents

The `metadata.csv` file provides a table of minimal metadata that describes the
layers in the collection. This metadata is intended to be sufficient for
users to find potential layers of interest, and make an initial
evaluation of their high-level characteristics such as taxonomic
details, spatial coverage and resolution, and model output types. Our
metadata is NOT intended to provide a comprehensive description of each
layer: users are referred to the original publication for that level of
detail (see the layer’s `reference` entry).

license: Layers are re-released under their original license where applicable, or CC-BY otherwise; individual items may have varying conditions — consult the 'licence' field in metadata.csv.

providers: role [processor, host] for 'SCAR Expert Group on Biodiversity Informatics' is an AI guess — Compiles and republishes model outputs from multiple original authors; acts as aggregator/processor rather than primary producer.. Verify before flipping to _state: reviewed.

providers: role [producer] for 'Royal Belgian Institute of Natural Sciences (ADVANCE / RBINS)' is an AI guess — Listed as a project partner and contributor; assumed co-producer of some model outputs.. Verify before flipping to _state: reviewed.

providers: role [producer] for 'Australian Antarctic Division' is an AI guess — Listed as a project partner via the Integrated Digital East Antarctica program; assumed co-producer of some model outputs.. Verify before flipping to _state: reviewed.


HIFLD Archive

An archive of the U.S. Homeland Infrastructure Foundation-Level Data (HIFLD) program, preserving hundreds of federal infrastructure datasets discontinued in August 2025. Coverage spans administrative boundaries, hydrography, transportation networks, critical infrastructure facilities, wildfire perimeters, and maritime wrecks across the United States and territories. All datasets have been converted to cloud-optimized Parquet format and spatially sorted by geohash level 8.

critical-infrastructure administrative-boundaries hydrology wildfire conservation maritime emergency-management homeland-security

Homeland Infrastructure Foundation-Level Data (HIFLD) Program

CENTLAT

[latlon] Centroid latitude of congressional district

CENTLON

[latlon] Centroid longitude of congressional district

GEOID

[admin_code] Census GEOID for the district

INTPTLAT

[latlon] Interior point latitude of congressional district

INTPTLON

[latlon] Interior point longitude of congressional district

STATE

[admin_code] State code for the congressional district

geometry

[geometry] Congressional district boundary polygon

bbox

STRUCT(xmin DOUBLE, ymin DOUBLE, xmax DOUBLE, ymax DOUBLE)

geometry

[geometry] Catchment polygon boundary

[geometry] Flowline linear geometry

STRUCT(county VARCHAR, date DOUBLE, job_date BIGINT, state VARCHAR)

[geometry] Hydrolocation point geometry

properties

STRUCT(OBJECTID BIGINT, featuredate BIGINT, featuretype BIGINT, featuretypelabel VARCHAR, gnisid VARCHAR, gnisidlabel VARCHAR, id3dhp VARCHAR, mainstemid VARCHAR, universalreferenceid VARCHAR, workunitid VARCHAR)

2 files sampled for this table

geometry

[geometry] Watershed boundary line geometry

[latlon] Latitude of fire discovery point

[latlon] Longitude of fire discovery point

[placename] City of point of origin for the fire

[placename] County of point of origin for the fire

[admin_code] FIPS code of the point-of-origin county

[placename] State of point of origin for the fire

[geometry] Wildfire perimeter polygon

[geometry] Wreck area polygon geometry

[geometry] Wreck or obstruction geometry

[geometry] Wreck point geometry

Source Cooperative product page

## README.md
# Homeland Infrastructure Foundation-Level Data (HIFLD) Archive

The Homeland Infrastructure Foundation-Level Data (HIFLD) program was a trusted resource for infrastructure data across the U.S. It supported policy, emergency management, and research before being discontinued in August 2025. This archive preserves that resource.

The datasets were processed using SeerAI Boson, Databricks, and Apache Sedona and saved in the Parquet format for ease of access directly on cloud object storage. If a dataset included geometry, it was spatially sorted using geohash level 8, which was then removed from the dataset.

## Dataset Directory Structure
Many of the original datasets contained multiple collections, and each collection was archived as its own parquet file. To preserve some of the original structure, there is a nested folder hierarchy for datasets and files. Please see below for the structure.

```
HIFLD
├── README.md                                       -> This file
├── hifld-dataset-inventory.csv                     -> Inventory of all datasets, their descriptions, any parent dataset folder, bucket location, and URL to vector tile JSON
├── 119th-congressional-districts                   -> Dataset parent folder
│   ├── README.md                                   -> Parent dataset README
│   ├── 119th-congressional-districts               -> Dataset folder
│   │   ├── 119th-congressional-districts.parquet   -> Dataset parquet
│   │   ├── README.md                               -> Dataset-specific README
├── 3d-hydrography-program-3dhp-layers              -> Dataset parent folder
│   ├── README.md                                   -> Parent dataset README
│   ├── catchment                                   -> Dataset folder - corresponding to collection in original dataset
│   │   ├── catchment.parquet                       -> Dataset parquet
│   │   ├── README.md                               -> Dataset-specific README
│   ├── flowline
│   │   ├── flowline.parquet
│   │   └── README.md
│   ├── ...
├── ...
```

## License
The HIFLD archive provided by SeerAI is released under the Creative Commons Attribution 4.0 International (CC BY 4.0).

## Citation
SeerAI, Inc. (2025). HIFLD Archive. Retrieved from https://www.seer.ai/

## Contact
For questions, issues, or feedback regarding this archive, please contact us at [contact@seerai.space](mailto:contact@seerai.space).


Apache Parquet Testing Data

This product contains a subset of the Parquet files published in https://github.com/apache/parquet-testing. It includes both correct (data) and bad (bad_data) Parquet files. The data/geospatial subdirectory contains test files for the new GEOMETRY logical type.


CSV PapaParse test files

The CSV files used to test the PapaParse library.


AMX Project 2024-04 PMTiles

The AMX (Adopt Map XML) Project converts open cadastral map data released by the Japanese Ministry of Justice into modern web formats. This dataset is derived from the 3rd release of 2024-04 and distributed as a single PMTiles archive covering Japan. It enables browser-based vector tile rendering of official Japanese land parcel map data.

land-parcels cadastral pmtiles ministry-of-justice

Dataset files (1 x .pmtiles)

Source Cooperative product page

providers: role [processor] for 'smartmaps' is an AI guess — Assumed processor role based on dataset being hosted under the 'smartmaps' account and described as produced from the Ministry of Justice source data.. Verify before flipping to _state: reviewed.


Japan GSI 1m GSD Terrain Tiles in PMTiles

High-resolution terrain tiles for Japan at 1 meter ground sample distance (GSD), produced by the Geospatial Information Authority of Japan (GSI) and packaged as a single PMTiles archive. Covers Japan's terrain with elevation data derived from official government survey sources.

topography elevation digital-elevation-model pmtiles remote-sensing

Geospatial Information Authority of Japan (GSI / 国土地理院)

Dataset files (1 x .pmtiles)

Source Cooperative product page


High-resolution 1-meter mesh Digital Elevation Model (DEM) tiles for Japan, derived from the Geospatial Information Authority of Japan's (GSI) Foundation Map Information dataset. Distributed as a single PMTiles archive for web-based terrain visualization and elevation analysis.

elevation topography digital-elevation-model pmtiles

Geospatial Information Authority of Japan (国土地理院)

Dataset files (1 x .pmtiles)

Source Cooperative product page

## README.md
# dem1a: PMTiles terrain tiles from dem1a by Geospatial Information Authority of Japan
## Demo

https://observablehq.com/d/c0a3f48cf7111cd2

## Source

基盤地図情報（数値標高モデル）１ｍメッシュ（標高）

測量法に基づく国土地理院長承認（使用）R 6JHs 133


FOIL4G Data Pack Release 1

FOIL4G (Free and Open Information Library for Geospatial) Data Pack Release 1 is a collection of seven global-extent PMTiles files covering the full Web Mercator range. The pack totals approximately 25 GB and appears to be a curated release of open geospatial tile data.

Dataset files (7 x .pmtiles)

Source Cooperative product page

providers: role [producer, licensor] for 'FOIL4G' is an AI guess — No README; assumed producer/licensor from the project name (Free and Open Information Library for Geospatial). Verify before flipping to _state: reviewed.


Smart Maps Global Elevation Tiles

Global RGB elevation tiles derived from NASADEM (zoom levels 6–12) and Global Map/ISCGM data (zoom levels 2–5), packaged as a single PMTiles file using Mapbox RGB pixel-packing (WebP). The dataset covers nearly the entire globe and is released as open data to reduce copyright constraints on terrain visualization. Produced under the UN Smart Maps Group of the UN Open GIS Initiative.

elevation topography digital-elevation-model pmtiles remote-sensing global-coverage raster-tiles

ISCGM / National Geospatial Information Authorities (Global Map)

Dataset files (1 x .pmtiles)

Source Cooperative product page

## README.md
## Overview

This dataset is RGB elevation tiles of the world converted from open data namely NASADEM (zoom level: 6-12) and Global Map data (zoom level: 2-5). Although we can find RGB elevation tiles dataset easily, sometimes it is difficult to use it because of the copyright. We made these RGB elevation tiles as open data to reduce constraints in the importation of geographical data and make the imported data flexible and effectively usable anywhere. This work is done under the UN Smart Maps Group of UN Open GIS Initiative.

## Demo: See it in action

- [A demo on Observable](https://observablehq.com/d/652ef8ca26776146) for a quick overview of the dataset.
- [A fullscreen website](https://optgeo.github.io/glob/) with hash URL enabled.

## Original datasets

- NASADEM

NASADEM is based on SRTM and complemented by ASTER G-DEM and ALOS 30 meter DEM. It has 1 arc second spatial resolutions (about 30 meters).

https://lpdaac.usgs.gov/products/nasadem_hgtv001/

- Global Map (ISCGM)

Global Map is fundamental digital geospatial information covering the whole land of the globe. Global Map data were developed under the cooperation of National Geospatial Information Authorities (NGIAs) of respective countries and regions. ISCGM served as the secretariat until the project concluded in 2017.

https://globalmaps.github.io/

## Data

Format: PMTiles

Zoom level: 6 to 12

Spatial Extent:
- ZL 2-5 Global (around 85°N to 85°S, 180°W to 180°E)
- ZL 6-12 Global (60°N to 56°S, 180°W to 180°E)

Size: 182.5 GB (The number of file: 1)

Convert Date: 2022 October

URL: https://data.source.coop/smartmaps/gel/gel.pmtiles

## Dataset License

CC0(No Rights Reserved)

https://creativecommons.org/public-domain/cc0/

## Reference

- Technical Detail

*https://qiita.com/T-ubu/items/c7bb5697b51f245ef163*

*https://qiita.com/T-ubu/items/ced8423e0ebeda8c9244*

- UN Smart Maps Group website (on GitHub Pages)

https://github.com/UNopenGIS/7

- UN Open GIS Initiative website

http://unopengis.org/unopengis/main/main.php

providers: role [processor] for 'UN Smart Maps Group / UN Open GIS Initiative' is an AI guess — README credits this group with converting the source DEMs to RGB PMTiles; assumed processor role.. Verify before flipping to _state: reviewed.


H3YS WorldPop

WorldPop unconstrained population counts for individual countries from 2000 to 2020, originally at 100m resolution, reaggregated into H3 hexagonal cells and distributed as cloud-optimized PMTiles. Part of the UN Open GIS Initiative Smart Maps Portable Web project for lightweight, portable web mapping.

population-counts demographics worldpop h3 pmtiles temporal-series

Dataset files (11 x .pmtiles)

Source Cooperative product page

## README.md
# h3ys-worldpop
WorldPop Population Counts / Unconstrained individual countries 2000-2020
(originally 100m resolution) aggregated in H3 and packed in PMTiles.

This is a UN Open GIS Initiative Smart Maps Portable Web project.

# Demo
https://hfu.github.io/worldpop-h3

# Development
https://github.com/hfu/worldpop-h3

# Issue and Todo tracking
https://github.com/unopengis/7/issues/652

# See Also
https://hub.worldpop.org/geodata/listing?id=29


Japan GeoTIFF DEM for Mapterhorn

GeoTIFF DEM converted from 10m / 5m / 1m resolution DEM by Geospatial Information Authority of Japan (GSI). Approval for Reproduction pursuant to the Survey Act, granted by the Director-General of the Geospatial Information Authority of Japan (R8JHf51).

When reproducing this product in order to make it available to the general public, approval from the Director-General of the Geospatial Information Authority of Japan must be obtained.


Japan GSI seamlessphoto

Complete nationwide aerial imagery of Japan at maximum resolution (Zoom 18)
from the Geospatial Information Authority of Japan (GSI) seamlessphoto dataset,
packaged as PMTiles / MBTiles archive for efficient web distribution and processing.


Mobility Data GTFS PMTiles

A global PMTiles vector tile file derived from all active GTFS (General Transit Feed Specification) feeds in the Mobility Database Catalog, maintained by Mobility Data. It consolidates public transit network data—stops, routes, and shapes—from transit agencies worldwide into a single tileset. The dataset is described as a work in progress with planned updates.

public-transit gtfs pmtiles vector-tiles transit-networks mobility

Dataset files (1 x .pmtiles)

Source Cooperative product page

## README.md
# Mobility Data GTFS PMTiles

A single PMTiles file created from all active GTFS data from Mobility Database Catalog from Mobility Data.

This is a work in progress. Data will be updated.

## Demo
https://protomaps.github.io/PMTiles/?url=https%3A%2F%2Fdata.source.coop%2Fsmartmaps%2Fmobility-gtfs-pmtiles%2Fmobility.pmtiles

## Specifications
Downloading and conversion script is at https://github.com/optgeo/mmx-gtfs.

## See also
- https://mobilitydatabase.org/

providers: role [producer] for 'Mobility Data' is an AI guess — Operates the Mobility Database Catalog that aggregates GTFS feeds; underlying feeds are from individual transit agencies not separately attributed in the README.. Verify before flipping to _state: reviewed.

providers: role [processor] for 'optgeo' is an AI guess — Assumed processor based on authorship of the mmx-gtfs download and conversion script linked in the README.. Verify before flipping to _state: reviewed.


Nagasaki LiDAR 3D Tiles (Ver. 2024-05)

LiDAR-derived 3D point cloud dataset covering a section of Nagasaki, Japan, delivered in the 3D Tiles .pnts format. The dataset contains over 2,100 point cloud tile files spanning the geographic extent of the surveyed area. Version dated May 2024.

lidar point-cloud 3d-tiles urban-mapping building-footprints

Dataset files (2142 x .pnts)

Source Cooperative product page


国土数値情報サンプル

Sample files from Japan's National Land Numerical Information (国土数値情報, KSJ) published by Japan's Ministry of Land, Infrastructure, Transport and Tourism (MLIT). Includes 2024 land price survey points, 2024 administrative boundary polygons, and 2006–2009 river line data covering Japan. Distributed in PMTiles and FlatGeobuf formats.

administrative-boundaries land-price rivers hydrology national-land-information open-government-data

Japan Ministry of Land, Infrastructure, Transport and Tourism (MLIT) GIS Lab

Dataset files (3 x .fgb)

Source Cooperative product page

## README.md
# next-ksj
https://github.com/gislab-mlit/next-ksj-formats で公開されているサンプルファイルのコピーです。
PMTiles 形式と FlatGeobuf 形式のファイルのコピーを置いています。

# pmtiles.io による閲覧
## L01-24_地価公示データ_2024年(令和6年)版
- https://pmtiles.io/?url=https%3A%2F%2Fdata.source.coop%2Fsmartmaps%2Fnext-ksj%2FL01-24.pmtiles#map=11.64/35.6818/139.7465

## N03-20240101_行政区域データ_2024年(令和6年)版 （ポリゴン）
- https://pmtiles.io/?url=https%3A%2F%2Fdata.source.coop%2Fsmartmaps%2Fnext-ksj%2FN03-20240101.pmtiles#map=4.21/33.91/138.46

## W05_河川データ_2009年度～2006年度(平成21年度～平成18年度) （ライン）
- https://pmtiles.io/?url=https%3A%2F%2Fdata.source.coop%2Fsmartmaps%2Fnext-ksj%2FW05-Stream.pmtiles#map=9.64/35.7108/139.8437

# See Also
https://github.com/gislab-mlit/next-ksj-formats/issues/18


Smart Maps LiDAR 3D Tiles from Open Nagasaki

A 3D Tiles point cloud dataset assembled by the UN Smart Maps Group from all publicly available LiDAR data published through Open Nagasaki (opennagasaki.nerc.or.jp). The dataset covers the Nagasaki prefecture region of Japan and is served in the .pnts (3D Tiles Point Cloud) format, comprising over 1.7 million tiles totalling approximately 232 GB.

Dataset files (1715716 x .pnts)

Source Cooperative product page

providers: role [producer, licensor] for 'Open Nagasaki (NERC)' is an AI guess — Original LiDAR data source; licensor assumed as the originating open-data portal. Verify before flipping to _state: reviewed.


OpenCellid PMTiles - FOIL4G (Free and Open Information Library for Geospatial)

OpenCelliD is the world's largest collaborative community project collecting GPS positions of cell towers globally. This dataset packages the full OpenCelliD cell tower database as a PMTiles file for efficient spatial tile access. It covers cell tower locations worldwide across multiple radio access technologies and network operators.

cell-towers telecommunications mobile-networks crowd-sourced coverage-mapping pmtiles

Dataset files (1 x .pmtiles)

Source Cooperative product page

providers: role [processor] for 'FOIL4G' is an AI guess — Assumed processor role based on the dataset title crediting FOIL4G for the PMTiles conversion of the original CSV source. Verify before flipping to _state: reviewed.


rw-wip-2024-05-31


Terrain tiles in PMTiles from Dem_Rwanda_10m_allt_20230921150153_band1.tif

Terrain tiles in PMTiles from https://geohub.data.undp.org/data/00d5add9be37e465398b081683c3ec03#Info. The original data is by Water and Sanitation Corporation (WASAC), National Land Authority (NLA).


Study on Strategic Use of Geospatial Information; support for JICA

A UN Smart Maps Group repository in support of JICA Study on Strategic Use of Geospatial Information (2025-04/2025-11)


Topographic Maps of East Timor (TMET)

This is an old topographic map dataset for East Timur created with UNTAET.

JICA and UNTAET agreed that the digital topographic maps and digital data for GIS to be prepared by the Study on Urgent Establishment of Topographic Mapping East Timor will be open to the pubic use.

However, this repository is not currently intended for public release of the data.


都市計画決定PMTiles

Urban planning decision GIS data from Japan's Ministry of Land, Infrastructure, Transport and Tourism (MLIT), covering 21 urban planning zone categories including land use zones (用途地域), urban planning areas, road networks, parks, and fire prevention districts. Derived from official MLIT sources and served as a single PMTiles vector tile file spanning the Japanese archipelago.

Ministry of Land, Infrastructure, Transport and Tourism (MLIT), Japan

Dataset files (1 x .pmtiles)

Source Cooperative product page

## README.md
# tk
都市計画決定GISデータ（国土交通省）

## Demo
- [PMTiles Viewer](https://protomaps.github.io/PMTiles/?url=https%3A%2F%2Fdata.source.coop%2Fsmartmaps%2Ftoshik%2Fa.pmtiles#map=6.92/35.722/138.52)

## Source Specifications
https://www.mlit.go.jp/toshi/tosiko/content/001609328.pdf

## basic schema
- code - YoutoID (youto), youtoID (tkbt), and kubunID unified.
- name - 名称 and tokeiname unified.
- date - 決定日
- status - 決定区分
- by - 決定者
- no - 告示番号
- city - citycode (5桁)

## layers
1. youto 用途地域
2. tokei 都市計画区域
3. senbiki 区域区分
4. ritteki 立地適正化計画
5. douro 都市計画道路
6. bouka 防火準防火
7. koudoti 高度地区
8. koudori 高度利用地区
9. tochiku 土地区画整理事業
10. chikukei 地区計画
11. tkbt 特別用途地区
12. kouen 公園
13. tokureiyouseki 特例容積率適用地区
14. kousoujyukyo 高層住居誘導地区
15. kyojyuchosei 居住調整地域
16. tokuteiyuudou 特定用途誘導地区
17. tokuteibouka 特定防火街区整備地区
18. rekifuu 歴史的風土保存地区、歴史的風土特別保存地区
19. ryokukachiiki 緑化地域
20. soubou 航空機騒音障害防止地区、航空機騒音障害防止特別地区
21. fukkousaiseikyoten 一団地の復興再生拠点市街地形成施設

## code list
### youto
- 1 第１種低層住居専用地域
- 2 第２種低層住居専用地域
- 3 第１種中高層住居専用地域
- 4 第２種中高層住居専用地域
- 5 第１種住居地域
- 6 第２種住居地域
- 7 準住居地域
- 8 田園住居地域
- 9 近隣商業地域
- 10 商業地域
- 11 準工業地域
- 12 工業地域
- 13 工業専用地域

### senbiki
- 22 市街化区域
- 23 市街化調整区域

### ritteki
- 31 居住誘導区域
- 32 都市機能誘導区域

### douro
- 3011 道路
- 3023 広場

### bouka
- 24 防火地域
- 25 準防火地域

### tkbt
- 1 特別工業地区
- 2 文教地区
- 3 小売店舗地区
- 4 事務所地区
- 5 厚生地区
- 6 娯楽レクリエーション地区
- 7 観光地区
- 8 特別業務地区
- 9 中高層階住居専用地区
- 10 商業専用地区
- 11 研究開発地区

### kouen
- 3021 公園
- 3022 緑地
- 3024 墓園

### soubou
- 40 航空機騒音障害防止地区
- 41 航空機騒音障害防止特別地区

## attribution
[都市計画決定GISデータ（国土交通省）](https://www.mlit.go.jp/toshi/tosiko/toshi_tosiko_tk_000087.html)を加工して使用

## Data repository
[Source Cooperative smartmaps/toshik](https://beta.source.coop/repositories/smartmaps/toshik)

providers: role [processor] for 'smartmaps' is an AI guess — Inferred as processor from attribution stating MLIT data was processed (加工して使用); smartmaps publishes the converted PMTiles.. Verify before flipping to _state: reviewed.


PMTiles of UCDP (Uppsala Conflict Data Program) Georeferenced Event Dataset (GED) Global ver. 23.1

PMTiles vector tileset of the UCDP (Uppsala Conflict Data Program) Georeferenced Event Dataset (GED) Global version 23.1. The underlying dataset records individual organized violence events (state-based, non-state, and one-sided) worldwide from 1989 through 2022, each georeferenced to a point location. This tileset packages the data for efficient web-map delivery using the PMTiles archive format.

armed-conflict conflict-events political-violence organized-violence ucdp

Uppsala Conflict Data Program (UCDP), Uppsala University

Dataset files (1 x .pmtiles)

Source Cooperative product page


Vector Tiles (MVT) and 3D Tiles (b3dm) derived from Japan's Project PLATEAU 3D urban city models, covering areas from approximately 127°E–141°E, 26°N–43°N across Japan. The dataset conforms to Project PLATEAU Standard Product Specification V3 and contains nearly 2.5 million 3D model tiles alongside over 1.1 million vector map tiles.

Project PLATEAU / Ministry of Land, Infrastructure, Transport and Tourism (MLIT), Japan

Dataset files (2497447 x .b3dm)

Source Cooperative product page

## README.md
# xing: Vector Tiles and 3D Tiles from Project PLATEAU

## Source
[3D都市モデル（Project PLATEAU）ポータルサイト](https://www.geospatial.jp/ckan/dataset/plateau)

- 標準製品仕様書V3に対応したデータのみを収録することにします。


NASA CMR Metadata

No Description Provided


Source Cooperative Manifests

No Description Provided


sites.source.coop

Content for sites.source.coop landing page


Source Stats

Weekly CSVs with statistics about data hosted in Source Cooperative


USDA County Boundaries (Cloud-Native Geo Distribution)

USDA County Boundaries is the official county-level boundary file of the USDA (United States Department of Agriculture) Census of Agriculture 2017. This distribution of the dataset makes it available in cloud-native geospatial format (GeoParquet), and offers some additional transformations and processing to increase utility for agricultural modelling.

vector agriculture geoparquet usda county state


ESA WorldCereal

The ESA WorldCereal dataset provides farm point locations derived from 10-meter resolution global crop type classification maps, covering maize, winter cereals, and spring cereals. Points are partitioned by administrative boundaries (levels 0 and 1) and distributed in cloud-native GeoParquet format. The dataset contains nearly 7 million features spanning a near-global extent.

agriculture crop-classification remote-sensing maize cereals global-coverage arable-land land-use

Dataset files (1 x .geoparquet)

Source Cooperative product page


GDELT-SA

Daily-updated Parquet dataset of GDELT 2.0 geopolitical event records filtered to Saudi Arabia, partitioned by calendar day. Each row represents a news-reported event with GDELT actor and action codes, Goldstein scale scores, and sentiment metrics across three geographic reference frames (Actor1, Actor2, Action location). Records are deduplicated by source URL and pipeline-enriched with nearest-city geographic matching and optionally scraped article full text.

time-series event-data media-monitoring sentiment-analysis middle-east news-events conflict-monitoring

2026_01_01.parquet

[placename] Full place name associated with Actor 1

[admin_code] FIPS ADM1 code for Actor 1 location

[latlon] Latitude of Actor 1 geographic location (paired with Actor1Geo_Long)

[latlon] Longitude of Actor 1 geographic location (paired with Actor1Geo_Lat)

[placename] Full place name associated with Actor 2

[admin_code] FIPS ADM1 code for Actor 2 location

[latlon] Latitude of Actor 2 geographic location (paired with Actor2Geo_Long)

[latlon] Longitude of Actor 2 geographic location (paired with Actor2Geo_Lat)

[placename] Full place name of the event action location

[admin_code] FIPS country code for the action location

[admin_code] FIPS ADM1 (first-level administrative division) code for the action location

[latlon] Latitude of the event action location (paired with ActionGeo_Long)

[latlon] Longitude of the event action location (paired with ActionGeo_Lat)

NearestCity

[placename] Pipeline-enriched nearest city name matched to ActionGeo coordinates via Haversine distance

[placename] Pipeline-enriched nearest city name matched to Actor1 coordinates

[placename] Pipeline-enriched nearest city name matched to Actor2 coordinates

2026_01_02.parquet

2026_01_03.parquet

2026_01_04.parquet

2026_01_05.parquet

2026_06_13.parquet

2026_06_14.parquet

2026_06_15.parquet

2026_06_16.parquet

2026_06_17.parquet

Source Cooperative product page

## README.md
# GDELT Saudi Arabia Daily News Events

Enriched [GDELT 2.0](https://www.gdeltproject.org/) event records related to Saudi Arabia, updated daily. Each record is deduplicated, geo-enriched with nearest-city matching, and includes scraped article full text when available.

**Demo:** [geonews.tabaqat.dev](https://geonews.tabaqat.dev/) - Interactive map to explore the data.

## Bucket Structure

```
s3://us-west-2.opendata.source.coop/tabaqat/gdelt-sa/
└── country=SA/
    └── year=YYYY/
        ├── YYYY_MM_DD.parquet
        ├── YYYY_MM_DD.parquet
        └── ...
```

One Parquet file per day (ZSTD compressed). New files are synced daily around midnight UTC via GitHub Actions.

## What's in Each File

Every row is a single GDELT event. Columns fall into four groups:

| Group                | Examples                                                                                | Count |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------- | ----- |
| **Event**            | `GLOBALEVENTID`, `SQLDATE`, `EventCode`, `QuadClass`, `GoldsteinScale`, `AvgTone`, `NumMentions`                       | 34    |
| **Actor geography**  | `Actor1Geo_FullName`, `Actor1Geo_Lat/Long`, `Actor2Geo_*`                                                              | 16    |
| **Action geography** | `ActionGeo_FullName`, `ActionGeo_CountryCode`, `ActionGeo_Lat/Long`                                                    | 8     |
| **Geo enrichment**   | `NearestCity`, `CityPopulation`, `DistanceToCity_km`, `CoordQuality` (×3 geo sets)                                     | 12    |
| **Article**          | `ArticleTitle`, `ArticleContent`, `ArticleAuthor`, `ArticlePublishDate`, `ArticleContentLength`, `ArticleScrapeMethod` | 6     |

**~76 columns total.** Full schema details in the [source repo](https://github.com/tabaqatdev/gdelt-cng).

## Processing Pipeline

Raw GDELT 15-minute dumps go through:

1. **Filter** — keep only events involving Saudi Arabia
2. **Clean** — remove nulls, empty URLs, invalid tone values
3. **Deduplicate** — one best record per source URL (scored by data completeness)
4. **Geo-enrich** — match coordinates to nearest city using a 33K+ world cities reference via Haversine distance
5. **Scrape** — extract article text using Trafilatura → Newspaper4k → Playwright fallback chain (up to 500/day)

## Quick Query (DuckDB)

```sql
-- read a single day
SELECT * FROM 's3://us-west-2.opendata.source.coop/tabaqat/gdelt-sa/country=SA/year=2026/2026_01_01.parquet' LIMIT 10;

-- count events with articles across all days
SELECT count(*) AS total,
       count(ArticleTitle) AS with_articles
FROM 's3://us-west-2.opendata.source.coop/tabaqat/gdelt-sa/country=SA/**/*.parquet';

-- top mentioned cities
SELECT NearestCity, count(*) AS n
FROM 's3://us-west-2.opendata.source.coop/tabaqat/gdelt-sa/country=SA/**/*.parquet'
WHERE NearestCity IS NOT NULL
GROUP BY 1 ORDER BY 2 DESC LIMIT 10;
```

## License & Attribution

Data derived from the [GDELT Project](https://www.gdeltproject.org/). Refer to GDELT's terms of use for redistribution rights. Pipeline source code is MIT licensed.

extent.spatial.bbox: omitted; spatial reference is categorical via [01.ActionGeo_Lat (latlon), 01.ActionGeo_Long (latlon), 01.ActionGeo_FullName (placename), 01.ActionGeo_CountryCode (admin_code), 01.ActionGeo_ADM1Code (admin_code), 01.Actor1Geo_Lat (latlon), 01.Actor1Geo_Long (latlon), 01.Actor1Geo_FullName (placename), 01.Actor1Geo_ADM1Code (admin_code), 01.Actor2Geo_Lat (latlon), 01.Actor2Geo_Long (latlon), 01.Actor2Geo_FullName (placename), 01.Actor2Geo_ADM1Code (admin_code), 01.NearestCity (placename), 01.Actor1_NearestCity (placename), 01.Actor2_NearestCity (placename)]. Add bbox once the placename → bbox gazetteer covers it.


geocoding-cng

A geocoding dataset for Saudi Arabia containing over 35,000 address-level records organized into H3 spatial index tiles, covering street addresses with full administrative hierarchy (district, governorate, region) in both Arabic and English. Includes companion polygon boundary layers for Saudi regions, governorates, and districts, plus a postcode index and a world countries reference layer.

geocoding addresses administrative-boundaries postcode h3 arabic

postcode_index.parquet

postcode

[zipcode] Saudi postal code used as lookup key for H3 tile list

[placename] Region name in Arabic

[placename] Region name in English

region_index.parquet

[placename] Region name in Arabic with bounding box

sa_districts_simple.parquet

city

[placename] City the district belongs to

centroid

[geometry] Centroid point of the district polygon

geometry

[geometry] Polygon boundary of the district

sa_governorates_simple.parquet

[geometry] Polygon boundary of the governorate

centroid

[geometry] Centroid point of the governorate polygon

sa_regions_simple.parquet

[geometry] Centroid point of the region polygon

geometry

[geometry] Polygon boundary of the region

8553ad1bfffffff.parquet

longitude

[latlon] Address longitude (representative tile; same schema applies to all H3-named tiles)

latitude

[latlon] Address latitude

postcode

[zipcode] Postal code of the address

[placename] District name in Arabic

city

[placename] City name

[placename] Region name in Arabic

[spatial_index] H3 cell index identifying the tile this address belongs to

8553ad63fffffff.parquet

8553ad67fffffff.parquet

8553ad6ffffffff.parquet

world_countries_simple.parquet

[admin_code] ISO 3166-1 alpha-3 country code

[admin_code] ISO 3166-1 alpha-2 country code

geometry

[geometry] Country polygon boundary

Source Cooperative product page

providers: role [producer] for 'CNG' is an AI guess — Inferred as producer from the dataset title 'geocoding-cng'; no README or explicit attribution available.. Verify before flipping to _state: reviewed.

extent.spatial.bbox: omitted; spatial reference is categorical via [postcode_index.postcode (zipcode), postcode_index.region_ar (placename), postcode_index.region_en (placename), region_index.region_ar (placename), sa_districts_simple.geometry (geometry), sa_districts_simple.centroid (geometry), sa_districts_simple.city (placename), sa_governorates_simple.geometry (geometry), sa_governorates_simple.centroid (geometry), sa_regions_simple.geometry (geometry), sa_regions_simple.centroid (geometry), 8553ad1bfffffff.longitude (latlon), 8553ad1bfffffff.latitude (latlon), 8553ad1bfffffff.postcode (zipcode), 8553ad1bfffffff.district_ar (placename), 8553ad1bfffffff.city (placename), 8553ad1bfffffff.region_ar (placename), 8553ad1bfffffff.h3_index (spatial_index), world_countries_simple.geometry (geometry), world_countries_simple.iso_a3 (admin_code), world_countries_simple.iso_a2 (admin_code)]. Add bbox once a placename → bbox gazetteer is available.


Overture Maps - Places Dataset (GeoParquet) - Riyadh, Saudi Arabia

A collection of 28,637 places (areas of interest) across Riyadh, Saudi Arabia, derived from Overture Maps and combining remote-sensed data from Microsoft and Meta corporations, provided in cloud-native GeoParquet format.

Dataset files (1 x .parquet)

Source Cooperative product page

## README.md

# Overture Maps - Places Dataset - Riyadh, Saudi Arabia

![Riyadh AOIs](https://i.imgur.com/g4vucn8.png)

# Overview
This dataset includes Overture Maps Places Data which is derived from Microsoft and Meta Corporations.
It contains 28,637 Places (AOIs) Data for Riyadh Region, Saudi Arabia. Last Updated On 11 February 2024.
The dataset was derived from Overture Maps By https://tabaqat.net/ in GeoParquet Format which is one of Cloud Native Geospatial Formats.

# Observe it in use
You can use https://youssef-harby.github.io/OvertureMapsDownloader/ to get a quick overview to see it in action through shared URL provided here https://source.coop/tabaqat/riyadh-places/riyadh_places.parquet by changing or removing bbox and features limit parameters.

# Files and Structure
- `riyadh_places.parquet` - geoparquet file of Riyadh AOIs dataset.
- `README.md` - This file, explaining the repository.

# Data Sources:
  This dataset was compiled by Overture Maps using data from Meta and Microsoft corporations.

# Data Formats:
the data is provided in this format:
- GeoParquet: which is an encoding for how to store geospatial vector data (point, lines, polygons) in Apache Parquet, a popular columnar storage format for tabular data.

# Schema:
This dataset adheres to Overture Maps Schema of Places theme explained here https://docs.overturemaps.org/schema/concepts/by-theme/places/

# Data Upload Process
Data Upload Process Steps are explained in our blog post https://blogs.tabaqat.net/en/unlocking-spatial-insights-a-guide-to-uploading-geoparquet-files-onto-source-cooperative-platform-for-enhanced-data-collaboration/

# Contact
For further details about the dataset or the processing steps, you can contact us via https://platform.tabaqat.net/contact-us

[https://github.com/sarah88932/GEE-Data-Script-files/blob/main/Riyadh%20AOIs-1.png]: https://github.com/sarah88932/GEE-Data-Script-files/blob/main/Riyadh%20AOIs-1.png


RGB satellite imagery of Riyadh Governorate, Saudi Arabia, derived from Sentinel-2 L2A data captured between July 1–14, 2024. Distributed as a single PMTiles archive suitable for web-based tile visualization. Processed from the Element84 Earth Search STAC catalog using the Tabaqat satellite tool.

remote-sensing satellite-imagery sentinel-2 pmtiles

Dataset files (1 x .pmtiles)

Source Cooperative product page

## README.md

# Sentinel-2 L2A RGB Satellite Imagery (PMTiles) - Riyadh Governorate, Saudi Arabia
![Riyadh Sentinel](https://i.imgur.com/saiubwz.png)
# Overview
This dataset includes (PMTiles) RGB Of Sentinel-2 L2A Satellite Imagery of Riyadh Governorate, Saudi Arabia Captured from (1 to 14) July 2024 ,
derived from https://earth-search.aws.element84.com/v1 by Tabaqat satellite tool
which is one of Tabaqat GeoApps tools provided by https://tabaqat.net/.

# Observe it in use
You can use https://pmtiles.io/ to get a quick overview to see it in action through shared URL provided here
https://data.source.coop/tabaqat/riyadh-satellite-pmtiles/riyadh_governorate.pmtiles at specified zoom extent.

# Files and Structure
- `riyadh_governorate.pmtiles` - PMTiles file of Riyadh Governorate.
- `README.md` - This file, explaining the repository.

# Data Sources:
  This dataset was compiled by tabaqat satellite tool from https://earth-search.aws.element84.com/v1.

# Data Formats:
the data is provided in this format:
- PMTiles is a single-file archive format for tiled data, usually used for visualization. As an “archive format”, PMTiles is similar to a ZIP file:
it contains the contents of many individual files inside of one PMTiles file. you can explore more information about PMTiles, from here https://guide.cloudnativegeo.org/pmtiles/intro.html.

# Data Access:
Generating PMTiles From Sentinel Satellite Imagery Produced By Tabaqat Satellite Tool is explained in our blog post https://tabaqat-wagtail.tabaqat.net/en/resources/blogs/generating-pmtiles-from-sentinel-satellite-imagery-produced-by-tabaqat-satellite-tool/.

# Data Upload Process
Data Upload Process Steps are explained in our blog post https://blogs.tabaqat.net/en/unlocking-spatial-insights-a-guide-to-uploading-geoparquet-files-onto-source-cooperative-platform-for-enhanced-data-collaboration/ which is valid for any data format.

# Contact
For further details about the dataset or the processing steps, you can contact us via https://platform.tabaqat.net/contact-us.

providers: role [producer] for 'Element84 (Earth Search)' is an AI guess — Credited as the data source (earth-search.aws.element84.com/v1); assumed producer role as the host of the source Sentinel-2 L2A STAC catalog.. Verify before flipping to _state: reviewed.


Sentinel-2 L2A RGB Satellite Imagery (COG) - Riyadh Governorate, Saudi Arabia

This repository include (Cloud Opimized Geotiff) Sentinel-2 L2A Satellite Imagery of Riyadh Governorate, Saudi Arabia from (1 to 14) July 2024 , derived from https://earth-search.aws.element84.com/v1 by Tabaqat satellite tool which is one of Tabaqat GeoApps tools provided by https://tabaqat.net/.

# You can navigate to @tabaqat to explore our repositories.

#sentinel #satellite #imagery #cog #aws #remotesensing #saudiarabia #tabaqat


Road network dataset for the Riyadh region of Saudi Arabia, containing 423,735 road features derived from Overture Maps (which sources from OpenStreetMap). The data is provided in PMTiles format, a cloud-native single-file archive suitable for tiled map visualization.

roads openstreetmap administrative-boundaries pmtiles

Dataset files (1 x .pmtiles)

Source Cooperative product page

## README.md

# Overture Maps - Roads Dataset (PMTiles) - Riyadh, Saudi Arabia

![Riyadh AOIs](https://i.imgur.com/Y8eQTP1.png)

# Overview
This dataset includes Overture Maps Roads Data which is retrieved from Open Street Maps Corporation.
It contains 423,735 Roads Features for Riyadh Region, Saudi Arabia. Last Updated On 12 February 2024.
The dataset was derived from Overture Maps By https://tabaqat.net/ in PMTiles Format which is one of Cloud Native Geospatial Formats.

# Observe it in use
You can use https://pmtiles.io/ to get a quick overview to see it in action through shared URL provided here https://data.source.coop/tabaqat/riyadh-overture-pmtiles/riyadh_riyadh_roads.pmtiles at specified zoom extent.

# Files and Structure
- `riyadh_roads.pmtiles` - PMTiles File of Riyadh Roads Dataset.
- `README.md` - This file, explaining the repository.

# Data Sources:
  This dataset was compiled by Overture Maps using data from Open Street Maps corporation.

# Data Formats:
the data is provided in this format:
- PMTiles is a single-file archive format for tiled data, usually used for visualization. As an “archive format”, PMTiles is similar to a ZIP file:
it contains the contents of many individual files inside of one PMTiles file. you can explore more information about PMTiles, from here https://guide.cloudnativegeo.org/pmtiles/intro.html.

# Data Access:
Generating PMTiles From Vector Geospatial Data is explained in our blog post https://tabaqat-wagtail.tabaqat.net/en/resources/blogs/generating-pmtiles-from-vector-geospatial-data-using-tippecanoe-and-go-pmtiles-tools/.

# Data Upload Process
Data Upload Process Steps are explained in our blog post https://blogs.tabaqat.net/en/unlocking-spatial-insights-a-guide-to-uploading-geoparquet-files-onto-source-cooperative-platform-for-enhanced-data-collaboration/

# Schema:
This dataset adheres to Overture Maps Schema of Places theme explained here https://docs.overturemaps.org/schema/concepts/by-theme/transportation/roads/

# Contact
For further details about the dataset or the processing steps, you can contact us via https://platform.tabaqat.net/contact-us

license: AI returned 'unknown' (README does not state a license). Verify with publisher before flipping to _state: reviewed.

providers: role [producer, licensor] for 'OpenStreetMap' is an AI guess — OSM is the ultimate data source; licensor role assumed based on OSM's ODbL governing derived datasets. Verify before flipping to _state: reviewed.

providers: role [processor] for 'Overture Maps Foundation' is an AI guess — Described as compiler of OSM data into the Overture Maps schema; processor role inferred. Verify before flipping to _state: reviewed.

providers: role [processor, host] for 'Tabaqat' is an AI guess — Derived the PMTiles file from Overture Maps data and uploaded to Source Cooperative; host role inferred from their platform presence. Verify before flipping to _state: reviewed.


Cloud 3D - GOES-16 Tropical Cyclones Finetuning Dataset

Cloud3DTACO is a large-scale AI-ready dataset pairing 2D multispectral geostationary imagery from GOES-16, Himawari-8/9, and MSG/SEVIRI with co-located 3D vertical cloud profiles from the CloudSat radar, comprising roughly 500,000 samples organized for self-supervised pretraining and supervised finetuning. This specific subset contains 121 GOES-16 finetuning samples focused on tropical cyclone structures (eyewalls and rainbands) derived from IBTrACS best-track records. Each sample consists of a geostationary imagery patch paired with an aligned CloudSat vertical profile label, both stored as Cloud-Optimized GeoTIFFs.

remote-sensing machine-learning satellite-imagery earth-observation meteorology radar tropical-cyclones cloud-microphysics

Dataset files (771242 x .tif)

Source Cooperative product page

## README.md

# A Global, AI-Ready Dataset for 3D Cloud Reconstruction

**Cesar Aybar**¹ · **Shirin Ermis**² · **Lilli Freischem**² · **Stella Girtsou**³⁴ · **Kyriaki-Margarita Bintsi**⁵
**Emiliano Diaz Salas-Porras**¹ · **Michael Eisinger**⁶ · **William Jones**² · **Anna Jungbluth**⁶ · **Benoit Tremblay**⁷

¹Universitat de València  ·  ²University of Oxford  ·  ³National Observatory of Athens  ·  ⁴National Technical University of Athens  ·  ⁵Harvard Medical School & Massachusetts General Hospital  ·  ⁶European Space Agency  ·  ⁷Environment and Climate Change Canada

---

## About

Cloud3DTACO pairs 2D multispectral geostationary imagery from **GOES-16**, **Himawari-8/9**, and **MSG/SEVIRI** with co-located 3D vertical cloud profiles from the **CloudSat** radar. The collection contains roughly 500,000 samples across 10 datasets, organized into a self-supervised **pretraining** stage (no labels) and a supervised **finetuning** stage (CloudSat profiles as ground truth). Each stage covers the three operational geostationary regions and includes a tropical-cyclone variant. The dataset is published as TACO v3 and follows FAIR principles.

## Datasets

| Subset | Pretraining (no labels) | Finetuning (CloudSat labels) |
|---|---|---|
| GOES-16 — Americas | [50,000 samples](https://source.coop/taco/3dclouds/pretraining/goes) | [31,046 samples](https://source.coop/taco/3dclouds/finetune/goes) |
| Himawari-8/9 — Asia & Oceania | [50,000 samples](https://source.coop/taco/3dclouds/pretraining/himawari) | [57,373 samples](https://source.coop/taco/3dclouds/finetune/himawari) |
| MSG/SEVIRI — Europe & Africa | [50,000 samples](https://source.coop/taco/3dclouds/pretraining/msg) | [181,653 samples](https://source.coop/taco/3dclouds/finetune/msg) |
| GOES — Tropical cyclones | [41,423 samples](https://source.coop/taco/3dclouds/pretraining/goes-cyclone) | [121 samples](https://source.coop/taco/3dclouds/finetune/goes-cyclone) |
| Himawari — Tropical cyclones | [38,711 samples](https://source.coop/taco/3dclouds/pretraining/himawari-cyclone) | [361 samples](https://source.coop/taco/3dclouds/finetune/himawari-cyclone) |
| **Total** | **230,134** | **270,554** |

## Storage

- **Pretraining** — each sample is a single multi-band Cloud-Optimized GeoTIFF at `DATA/{idx}.tif`.
- **Finetuning** — each sample is a folder with two COGs: `DATA/{idx}/geo_patch.tif` (geostationary imagery) and `DATA/{idx}/cloudsat_aligned.tif` (vertical cloud profile labels).

## Notebooks

- [**Demo 1 — Dataset Exploration & Visualization**](https://colab.research.google.com/drive/1ecZF5p_A_BkBi_G0LtizqeRgmNWFn2sq)
- [**Demo 2 — Self-Supervised Pretraining**](https://colab.research.google.com/drive/1z_xK7-Irp8Ohba_l3dtIGavRjQpxT3Zc)
- [**Demo 3 — Supervised Finetuning**](https://colab.research.google.com/drive/1P0CLVEGo-puhG2cHkpjc4BUooVm7nx4t)
- **Demo 4 — Pretraining + Finetuning** *(coming soon)*

## Acknowledgements

This work was enabled by the Frontier Development Lab (FDL) Earth Systems Lab, a public-private partnership between the European Space Agency (ESA), Trillium Technologies, and the University of Oxford. We thank the teams behind the GOES, Himawari, and Meteosat satellites for their invaluable data. This research was supported by computational resources from Google Cloud.


DarkTOM


MethaneSet


AEF-v1 embeddings Rasteret Collection

A read-ready Rasteret collection derived from the AlphaEarth Foundations (AEF) annual satellite embedding dataset produced by Google and Google DeepMind. It provides a partitioned GeoParquet index and per-tile Cloud Optimized GeoTIFF header metadata for approximately 700,000 annual tiles covering near-global land area, enabling remote spatial and temporal filtering before fetching 64-band embedding vectors from upstream COGs. The dataset stores no pixel data — only Parquet metadata structured for use with Rasteret, DuckDB, Polars, and PyArrow.

remote-sensing embeddings machine-learning earth-observation foundation-model geoparquet global-coverage satellite-imagery

data

[geometry] WGS84 tile geometry (OGC:CRS84) for each Rasteret collection record

assets

STRUCT(A00 STRUCT(band_index BIGINT, href VARCHAR), A01 STRUCT(band_index BIGINT, href VARCHAR), A02 STRUCT(band_index BIGINT, href VARCHAR), A03 STRUCT(band_index BIGINT, href VARCHAR), A04 STRUCT(band_index BIGINT, href VARCHAR), A05 STRUCT(band_index BIGINT, href VARCHAR), A06 STRUCT(band_index BIGINT, href VARCHAR), A07 STRUCT(band_index BIGINT, href VARCHAR), A08 STRUCT(band_index BIGINT, href VARCHAR), A09 STRUCT(band_index BIGINT, href VARCHAR), A10 STRUCT(band_index BIGINT, href VARCHAR), A11 STRUCT(band_index BIGINT, href VARCHAR), A12 STRUCT(band_index BIGINT, href VARCHAR), A13 STRUCT(band_index BIGINT, href VARCHAR), A14 STRUCT(band_index BIGINT, href VARCHAR), A15 STRUCT(band_index BIGINT, href VARCHAR), A16 STRUCT(band_index BIGINT, href VARCHAR), A17 STRUCT(band_index BIGINT, href VARCHAR), A18 STRUCT(band_index BIGINT, href VARCHAR), A19 STRUCT(band_index BIGINT, href VARCHAR), A20 STRUCT(band_index BIGINT, href VARCHAR), A21 STRUCT(band_index BIGINT, href VARCHAR), A22 STRUCT(band_index BIGINT, href VARCHAR), A23 STRUCT(band_index BIGINT, href VARCHAR), A24 STRUCT(band_index BIGINT, href VARCHAR), A25 STRUCT(band_index BIGINT, href VARCHAR), A26 STRUCT(band_index BIGINT, href VARCHAR), A27 STRUCT(band_index BIGINT, href VARCHAR), A28 STRUCT(band_index BIGINT, href VARCHAR), A29 STRUCT(band_index BIGINT, href VARCHAR), A30 STRUCT(band_index BIGINT, href VARCHAR), A31 STRUCT(band_index BIGINT, href VARCHAR), A32 STRUCT(band_index BIGINT, href VARCHAR), A33 STRUCT(band_index BIGINT, href VARCHAR), A34 STRUCT(band_index BIGINT, href VARCHAR), A35 STRUCT(band_index BIGINT, href VARCHAR), A36 STRUCT(band_index BIGINT, href VARCHAR), A37 STRUCT(band_index BIGINT, href VARCHAR), A38 STRUCT(band_index BIGINT, href VARCHAR), A39 STRUCT(band_index BIGINT, href VARCHAR), A40 STRUCT(band_index BIGINT, href VARCHAR), A41 STRUCT(band_index BIGINT, href VARCHAR), A42 STRUCT(band_index BIGINT, href VARCHAR), A43 STRUCT(band_index BIGINT, href VARCHAR), A44 STRUCT(band_index BIGINT, href VARCHAR), A45 STRUCT(band_index BIGINT, href VARCHAR), A46 STRUCT(band_index BIGINT, href VARCHAR), A47 STRUCT(band_index BIGINT, href VARCHAR), A48 STRUCT(band_index BIGINT, href VARCHAR), A49 STRUCT(band_index BIGINT, href VARCHAR), A50 STRUCT(band_index BIGINT, href VARCHAR), A51 STRUCT(band_index BIGINT, href VARCHAR), A52 STRUCT(band_index BIGINT, href VARCHAR), A53 STRUCT(band_index BIGINT, href VARCHAR), A54 STRUCT(band_index BIGINT, href VARCHAR), A55 STRUCT(band_index BIGINT, href VARCHAR), A56 STRUCT(band_index BIGINT, href VARCHAR), A57 STRUCT(band_index BIGINT, href VARCHAR), A58 STRUCT(band_index BIGINT, href VARCHAR), A59 STRUCT(band_index BIGINT, href VARCHAR), A60 STRUCT(band_index BIGINT, href VARCHAR), A61 STRUCT(band_index BIGINT, href VARCHAR), A62 STRUCT(band_index BIGINT, href VARCHAR), A63 STRUCT(band_index BIGINT, href VARCHAR))

bbox

[geometry] WGS84 bounding box struct (xmin/ymin/xmax/ymax) for each tile

STRUCT(image_width INTEGER, image_height INTEGER, tile_width INTEGER, tile_height INTEGER, dtype VARCHAR, "transform" DOUBLE[], predictor INTEGER, "compression" INTEGER, tile_offsets BIGINT[], tile_byte_counts BIGINT[], pixel_scale DOUBLE[], tiepoint DOUBLE[], nodata DOUBLE, samples_per_pixel INTEGER, planar_configuration INTEGER, photometric INTEGER, extra_samples INTEGER[])

8 files sampled for this table

[geometry] UTM-projected bounding box for each tile

[geometry] WGS84 bounding box struct for each index tile

geom

[geometry] WGS84 tile geometry (OGC:CRS84) in the narrow GeoParquet index

location

[placename] Location identifier or name associated with each tile

2 files sampled for this table

Source Cooperative product page

## README.md
# AEF v1 Annual Rasteret Collection

This dataset is a read-ready [Rasteret](https://github.com/terrafloww/rasteret) collection derived from the public AlphaEarth Foundations (AEF) annual index and enriched with Rasteret's per-band Cloud Optimized GeoTIFF header metadata.

It is intended for Rasteret users who want to skip the build/enrich step and start directly from a filterable Parquet index using Rasteret, DuckDB, Polars, or PyArrow.

## What this dataset contains

Each row corresponds to one AEF annual tile and includes:

- `index.parquet`: a narrow GeoParquet 1.1 index with `fid`, `crs`, `path`, `year`,
  `utm_zone`, `bbox_utm`, `bbox`, `geom`, and `location`
- `data/part-xxxxx.parquet`: read-ready Rasteret collection shards with canonical
  runtime columns `id`, `datetime`, `geometry`, `bbox`, `assets`, `proj:epsg`,
  `year`, `utm_zone`, plus `A00_metadata` through `A63_metadata`

This dataset does **not** store embedding pixel arrays inside Parquet. It stores metadata that lets Rasteret fetch pixels directly from the original COGs.

## Why this exists

The source AEF STAC-geoparquet is already public, but Rasteret normally needs one more step: COG header enrichment. This collection publishes that work once so users can:

- filter remotely before reading any pixels
- join the collection to their own labels, manifests, splits, or experiment metadata
- reuse the same read-ready index across notebooks, jobs, and teams

## License and attribution

This derived metadata index is licensed under **CC-BY-SA-4.0**. It contains no pixel data — only Parquet metadata and COG headers of the imagery.

This dataset was built from the public AlphaEarth Foundations Satellite Embedding Dataset published on Source Cooperative by Taylor Geospatial Engine Labs:

- Source Cooperative product page: https://source.coop/tge-labs/aef
- Source Cooperative AEF index used as metadata input: https://source.coop/tge-labs/aef/v1/annual/aef_index.parquet

The underlying AlphaEarth Foundations Satellite Embedding data is produced by Google and Google DeepMind and is licensed under **CC-BY-4.0**.
This Terrafloww dataset is derived from the Source Cooperative AEF index and the Source Cooperative-hosted AEF COGs.

When using this derived index or the underlying embeddings, include the required upstream attribution text:

> "The AlphaEarth Foundations Satellite Embedding dataset is produced by Google and Google DeepMind."

Reference links:

- Google Earth Engine dataset page: https://developers.google.com/earth-engine/datasets/catalog/GOOGLE_SATELLITE_EMBEDDING_V1_ANNUAL
- Google AEF GCS README: https://developers.google.com/earth-engine/guides/aef_on_gcs_readme
- Taylor Geospatial Engine article on the Source Cooperative release: https://tgengine.org/building-frictionless-geospatial-ai-making-alphaearth-foundations-embeddings-accessible/
- This index license (CC-BY-SA-4.0): https://creativecommons.org/licenses/by-sa/4.0/
- Underlying data license (CC-BY-4.0): https://creativecommons.org/licenses/by/4.0/

## Citation

Use this citation for the derived Terrafloww dataset:

```bibtex
@dataset{terrafloww_aef_v1_annual_rasteret_2026,
  author    = {Terrafloww Labs, Inc.},
  title     = {AEF v1 Annual Rasteret Collection},
  year      = {2026},
  publisher = {Terrafloww Labs, Inc.},
  url       = {https://source.coop/terrafloww/aef-v1-annual-rasteret},
  note      = {Derived Rasteret metadata index built from the Source Cooperative AEF index and Source Cooperative-hosted AlphaEarth Foundations COGs published by Taylor Geospatial Engine Labs}
}
```

Also acknowledge the upstream data source in accompanying text:

- Source Cooperative AEF product published by Taylor Geospatial Engine Labs: https://source.coop/tge-labs/aef
- AlphaEarth Foundations data producer: Google and Google DeepMind
- Required attribution: "The AlphaEarth Foundations Satellite Embedding dataset is produced by Google and Google DeepMind."

## Recommended access patterns

Source Cooperative data is stored on S3. Use `s3://` URIs for glob support and predicate pushdown.

### Rasteret

```python
import rasteret

collection = rasteret.load("s3://us-west-2.opendata.source.coop/terrafloww/aef-v1-annual-rasteret/data")

# Filter to your area and time range
sub = collection.subset(
    bbox=(-95.5, 38.4, -95.0, 38.8),
    date_range=("2024-01-01", "2024-12-31"),
)

# Read pixels — choose your output format
ds = sub.get_xarray(geometries=(-95.4, 38.5, -95.1, 38.7), bands=["A00", "A01"], geometry_crs=4326)
```

## Data layout

- `index.parquet`: narrow GeoParquet 1.1 metadata index for remote filtering
- `data/part-xxxxx.parquet`: read-ready Rasteret collection shards
- rows are clustered by `year`, `utm_zone`, and WGS84 spatial order
- Parquet layout is tuned for repeated filtered reads rather than maximum full-scan throughput

## Intended use

This dataset is for metadata filtering, selection, and read-time handoff into Rasteret. It is especially useful when you want to combine AEF tiles with your own Arrow-native metadata workflows before reading embeddings.

## Also available on

- Hugging Face: [terrafloww/aef-v1-annual-rasteret](https://huggingface.co/datasets/terrafloww/aef-v1-annual-rasteret)

## Limitations

- It depends on the upstream AEF COG URLs remaining available.
- It is optimized for Rasteret and Arrow-native tooling, not for storing embedding payloads in Parquet.
- It is a derived metadata index, not a replacement for the upstream source dataset.

extent.spatial.bbox: omitted; spatial reference is categorical via [data.geometry (geometry), data.bbox (geometry), ..geom (geometry), ..bbox (geometry), ..bbox_utm (geometry), ..location (placename)]. Add bbox once the placename → bbox gazetteer covers it.


Cloud Native Admin Boundaries

Global administrative boundaries covering all jurisdictional levels, distributed in Cloud Native Geo parquet format.

Dataset files (11 x .parquet)

Source Cooperative product page


AlphaEarth Foundations GeoZarr Mosaic

A global mosaic of Google AlphaEarth Foundations Satellite Embeddings stored as a Zarr v3 array, covering 2017–2025 at approximately 10m resolution at the equator. Each pixel encodes a 64-dimensional learned embedding derived from satellite imagery, projected in EPSG:4326 and spanning ±83° latitude. Annual time steps (9 years) enable temporal change detection and similarity search across the full globe.

remote-sensing satellite-embeddings embeddings machine-learning foundation-model global-coverage time-series

Dataset files (1 x .zarr)

Source Cooperative product page

## README.md
# AEF Mosaic

A global mosaic of [Google AlphaEarth Foundations Satellite Embeddings](https://developers.google.com/earth-engine/datasets/catalog/projects_sat-io_open-datasets_AEF_AlphaEarthFoundationsSatelliteEmbedding), covering 2017-2025 at ~0.00009° resolution. The dataset is projected in `EPSG:4326` with a fixed pixel size in degrees. This means that pixels cover ~10m on the ground at the equator, but less area at higher latitudes. For area-based analysis, consider reprojecting to an equal-area projection, [creating your own mosaic](https://github.com/geospatial-jeff/aef-mosaic) in a custom CRS, or using the [underlying COGs](https://source.coop/tge-labs/aef) which are already projected to UTM. This copy is not officially supported by Google.

This dataset implements the following Zarr conventions.  These Zarr conventions are not mature and may change.

| Convention | Description |
|------------|-------------|
| [proj:](https://github.com/zarr-experimental/geo-proj/blob/v1/README.md) | Coordinate reference system information |
| [spatial:](https://github.com/zarr-conventions/spatial/blob/v1/README.md) | Spatial coordinate and transform metadata |
| [geoemb:](https://github.com/geo-embeddings/embeddings-zarr-convention/blob/add-convention/README.md) | Metadata for geo-embeddings |

## License

This dataset is licensed under CC-BY 4.0 and requires the following attribution text: "The AlphaEarth Foundations Satellite Embedding dataset is produced by Google and Google DeepMind."

## Visualizations

### RGB Composite
PCA visualization of the embedding space projected to RGB:

### RGB PCA (Zoomed)
Detailed view showing embedding structure at higher resolution:

### Cosine Similarity
Similarity search using a query embedding:

### Change Detection
Temporal embedding differences highlighting landscape changes:

## Quick Start

```python
import zarr
import numpy as np

root = zarr.open_group(
  "s3://us-west-2.opendata.source.coop/tge-labs/aef-mosaic/",
  mode="r",
  storage_options={"anon": True}
)
embeddings = root["embeddings"]

# Access a spatial subset (year 0, all bands, y slice, x slice)
data = embeddings[0, :, 516000:516100, 832000:832100]

# De-quantize to float32
data_float = ((data / 127.5) ** 2) * np.sign(data)
```

## Xarray

```python
import xarray as xr

def fetch_point(ds, lat, lon, time=0):
    """Fetch AEF embedding for a single lat/lon point. Returns shape (64,)."""
    return ds.isel(time=time).sel(x=lon, y=lat, method="nearest")

def fetch_bbox(ds, bbox, time=0):
    """Fetch AEF embeddings for a bounding box. Returns shape (64, y, x)."""
    minx, miny, maxx, maxy = bbox
    return ds.isel(time=time).sel(x=slice(minx, maxx), y=slice(maxy, miny))

def main():
    ds = xr.open_zarr(
        "s3://us-west-2.opendata.source.coop/tge-labs/aef-mosaic/",
        storage_options={"anon": True}
    )

    # Fetch embedding for a single point
    lat, lon = 34.0522, -118.2437  # Los Angeles
    point_data = fetch_point(ds, lat, lon).compute()
    print(f"Point shape: {point_data.embeddings.shape}")

    # Fetch embeddings for a bounding box
    bbox = [-118.279668, 34.036242, -118.20862, 34.089643]
    bbox_data = fetch_bbox(ds, bbox).compute()
    print(f"Bbox shape: {bbox_data.embeddings.shape}")

if __name__ == "__main__":
    main()
```

## Zarr-python

```python
import asyncio
from affine import Affine
from obstore.store import S3Store
from zarr.storage import ObjectStore
from zarr.api.asynchronous import open_group

store = S3Store(
    "us-west-2.opendata.source.coop",
    prefix="tge-labs/aef-mosaic/",
    region="us-west-2",
    skip_signature=True,
)
zarr_store = ObjectStore(store, read_only=True)

def latlon_to_rowcol(lat, lon, transform):
    """Convert lat/lon to row/col using the inverse transform."""
    inv = ~transform
    col, row = inv * (lon, lat)
    return int(row), int(col)

def bbox_to_slices(bbox, transform):
    """Convert a bounding box to row/col slices."""
    minx, miny, maxx, maxy = bbox
    inv = ~transform
    col1, row1 = inv * (minx, maxy)
    col2, row2 = inv * (maxx, miny)
    row_start, row_stop = int(min(row1, row2)), int(max(row1, row2)) + 1
    col_start, col_stop = int(min(col1, col2)), int(max(col1, col2)) + 1
    return slice(row_start, row_stop), slice(col_start, col_stop)

async def fetch_point(embeddings, transform, lat, lon, year=0):
    """Fetch AEF embedding for a single lat/lon point. Returns shape (64,)."""
    row, col = latlon_to_rowcol(lat, lon, transform)
    return await embeddings.getitem((year, slice(None), row, col))

async def fetch_bbox(embeddings, transform, bbox, year=0):
    """Fetch AEF embeddings for a bounding box. Returns shape (64, rows, cols)."""
    row_slice, col_slice = bbox_to_slices(bbox, transform)
    return await embeddings.getitem((year, slice(None), row_slice, col_slice))

async def main():
    root = await open_group(store=zarr_store, mode="r")
    embeddings = await root.get("embeddings")
    transform = Affine(*root.metadata.attributes["spatial:transform"])

    # Fetch embedding for a single point
    lat, lon = 34.0522, -118.2437  # Los Angeles
    point_data = await fetch_point(embeddings, transform, lat, lon)
    print(f"Point shape: {point_data.shape}")

    # Fetch embeddings for a bounding box
    bbox = [-118.279668, 34.036242, -118.20862, 34.089643]
    bbox_data = await fetch_bbox(embeddings, transform, bbox)
    print(f"Bbox shape: {bbox_data.shape}")

if __name__ == "__main__":
    asyncio.run(main())
```

## Dataset Details

| Property | Value |
|----------|-------|
| **Location** | `s3://us-west-2.opendata.source.coop/tge-labs/aef-mosaic/` |
| **Format** | Zarr v3 with sharding |
| **Shape** | `(9, 64, 1859584, 4009984)` |
| **Dimensions** | `(time, band, y, x)` |
| **Years** | 2017-2025 (indices 0-8) |
| **Bands** | 64 embedding dimensions |
| **CRS** | EPSG:4326 |
| **Bbox** | `[-180.0, -83.36, 180.22, 83.69]` |
| **Resolution** | ~0.00009° (~10m at equator) |
| **Data Type** | int8 (quantized), nodata = -128 |
| **Shard Size** | `(1, 64, 4096, 4096)` |
| **Inner Chunk Size** | `(1, 64, 256, 256)` |

## Source Code

The processing pipeline used to create this mosaic is available at:
https://github.com/geospatial-jeff/aef-mosaic

## Contact

Please send any comments, questions, or points of feedback to the [`#opening-aef`](https://cloudnativegeo.slack.com/archives/C09A8SV3WN5) CNG slack channel.


AlphaEarth Foundations Satellite Embedding Dataset

Annual satellite embeddings derived from Google's Earth Engine Satellite Embedding dataset (V1), covering 2017–2025. Each year's data is organized by UTM zone as 8192×8192-pixel Cloud-Optimized GeoTIFFs with 64 embedding channels per pixel, encoding compact learned representations of the Earth's surface from satellite imagery. This copy was produced by tge-labs and is not officially supported by Google.

remote-sensing satellite-embeddings embeddings machine-learning google-earth-engine annual-time-series cloud-optimized-geotiff utm

Dataset files (302466 x .tiff)

Source Cooperative product page

## README.md
# AlphaEarth Foundations GCS data

This product contains COG (Cloud-Optimized GeoTIFF) files that together contain the AlphaEarth Foundations annual [Satellite Embedding dataset](https://developers.google.com/earth-engine/datasets/catalog/GOOGLE_SATELLITE_EMBEDDING_V1_ANNUAL). It contains the annual embeddings for the years from 2017 to 2025.  This copy is not officially supported by Google.

#### **Warning**
The COGs in this dataset are "bottom-up" where the origin of the image is the bottom-left corner, the y-resolution is positive, and image blocks are ordered from bottom-left to top-right in the file.  A standard COG is "top-down" where the origin is the top-left corner, the y-resolution is negative, and image blocks are ordered from top-left to bottom-right. While some viewers like QGIS are smart enough to flip the images, accessing the data directly with most software (like GDAL) may cause errors.  We've provided a corresponding `.vrt` along with each `.tiff` file in the bucket which is responsible for correcting this mistake on-the-fly.  Please use the provided VRTs to access the data for any use cases that go beyond simple visualization.

## License

This dataset is licensed under CC-BY 4.0 and requires the following attribution text: "The AlphaEarth Foundations Satellite Embedding dataset is produced by Google and Google DeepMind."

## Directory structure

They are divided into directories by year; each year's directory is divided into 120 subdirectories, one per UTM zone, whose names reflect the zone number and hemisphere (`N` or `S`).

Within each directory are a number of COG files. These files contain all the pixel data for that UTM zone.

## File structure

Each file is 8192x8192 pixels, with 64 channels. The magnitude of each pixel, after the de-quantization mapping has been applied (see below), has been normalized so that it has a Euclidean length of 1.

The files contain overview layers at 4096x4096 pixels, 2048x2048 pixels, and so on down to a 1x1 top-level overview layer. These overview layers are constructed so that each overview pixel is the mean of the highest-resolution pixels under that overview pixel, where the mean's magnitude has been normalized to have length 1.

The channels correspond, in order, to the `A00` through `A63` axes of the Satellite Embedding dataset. The COGs also contain this naming for the channels.

Each pixel's value for each channel is a signed 8-bit integer. The way in which these values are mapped to the native values (in [-1, 1]) of the embeddings is described below.

The value -128 corresponds to a masked pixel. If it is present in one channel, it will be present in all channels. The COGs reflect this (i.e., they have the `NoData` value set to -128).

The name of each file also carries some information. For, example, consider the file named
[https://source.coop/tge-labs/aef/v1/annual/2019/1S/x8qqwcsisbgygl2ry-0000008192-0000000000.tiff](https://source.coop/tge-labs/aef/v1/annual/2019/1S/x8qqwcsisbgygl2ry-0000008192-0000000000.tiff).

As described above, this file is part of the 2019 annual embedding, and is in UTM zone 1S (zone 1, southern hemisphere). The base filename, `x8qqwcsisbgygl2ry-0000008192-0000000000`, serves to link this file to the corresponding Earth Engine Satellite Embedding Image name. In this example, this file corresponds to a portion of the Earth Engine image `GOOGLE/SATELLITE_EMBEDDING/V1/ANNUAL/x8qqwcsisbgygl2ry`. The two decimal parts of the filename specify where this COG's values are relative to that Earth Engine Image, as an offset in Y followed by an offset in X. In this case, the COG's pixel origin is at (0, 8192) relative to the Earth Engine Image's origin.

This is because it was necessary to subdivide each Earth Engine Image (which are 16384x16384 pixels) so that the resulting COGs would not be too unwieldy.

## De-quantization

To transform the raw signed 8-bit value (which will be between -127 and 127 inclusive - -128 is reserved as the "no data" value) in each channel of each pixel to the analysis-ready floating-point value (which will be between -1 and 1), the mapping to perform is

- divide by 127.5
- square
- multiply by the sign of the original value

This would be expressed in NumPy as

```python
  # values is a NumPy array of raw pixel values
  de_quantized_values = ((values / 127.5) ** 2) * np.sign(values)
```

In Earth Engine, the corresponding operation would be

```javascript
  var de_quantized_values = values.divide(127.5).pow(2).multiply(values.signum());
```

## Manifest and index

A list of the files in this dataset can be found in
[https://source.coop/tge-labs/aef/v1/annual/manifest.txt](https://source.coop/tge-labs/aef/v1/annual/manifest.txt).

As it is not possible to determine from the file names what area of the world they cover, an index has also been provided, in three forms (GeoParquet, GeoPackage, and CSV) in the files
https://source.coop/tge-labs/aef/v1/annual/aef_index.parquet,
https://source.coop/tge-labs/aef/v1/annual/aef_index.gpkg, and
https://source.coop/tge-labs/aef/v1/annual/aef_index.csv. This
index contains one entry for each file in the dataset. The information provided for each file is:

- the geometry of the file, as a WGS84 (i.e., EPSG:4326) polygon. In the CSV form, this is in the column `WKT`. See below for details on how this geometry is computed.
- `crs`: The CRS of the UTM zone this image belongs to, as an EPSG code, like `EPSG:32610`.
- `year`: The year that the image covers.
- `utm_zone`: The UTM zone of the image, like `10N`.
- `utm_west`, `utm_south`, `utm_east`, `utm_north`: The UTM bounds of the raw pixel array. This does not reflect any geometry processing, and includes all pixels whether or not they are valid.
- `wgs84_west`, `wgs84_south`, `wgs84_east`, `wgs84_north`: The min/max longitude and latitude of the WGS84 geometry.

### Geometry processing

The pixel array is natively in some UTM zone, so in that UTM zone the bounding box of the pixel array is a simple rectangle. That bounding box is transformed into a polygon in WGS84. This polygon includes a number of extra points so that its edges closely follow the curved lines in WGS84 that the straight lines in UTM transform into. This polygon does not take into account the validity/invalidity of pixels in the image, just the bounds of the image's pixel array.

The polygon is then clipped to the minimum and maximum longitude of the image's UTM zone. In practice, this may cause it to not include a few valid pixels that hang over the edge of the UTM zone. Omitting these pixels from the index shouldn't cause any problems: some image from the neighboring UTM zone should cover that area.

Note that clipping to the min/max longitude of the UTM zone means that no polygon crosses the antimeridian, which should make processing this file a little simpler.

providers: role [processor, host] for 'tge-labs' is an AI guess — README states this is an unofficial copy; tge-labs assumed to be the processor who converted to COG format and the host.. Verify before flipping to _state: reviewed.


GlobalBuildingAtlas LoD1

Global dataset of approximately 2.75 billion building footprints in Level of Detail 1 (LoD1) format, combining data from Google Open Buildings, OpenStreetMap, Microsoft Building Footprints, and new detections from deep learning analysis of Planet Labs satellite imagery. Includes building geometry, height information, and geographic coordinates.

remote-sensing building-footprints building-heights urban-mapping 3d-models

Dataset files (922 x .parquet)

Source Cooperative product page

## README.md
This repository contains the [GlobalBuildingAtlas](https://mediatum.ub.tum.de/1782307) (GBA) Level of Detail 1 (LoD1) building footprints dataset, converted from the original 922 GeoJSON files into Parquet format for improved performance and accessibility.

## Dataset Overview

The GlobalBuildingAtlas LoD1 dataset contains building footprints for approximately 2.75 billion buildings worldwide. This represents the most comprehensive global building dataset available, covering nearly all inhabited areas on Earth.

The dataset combines multiple existing building footprint datasets with new footprints generated using open source deep learning models applied to Planet Labs satellite imagery.

This Parquet version was converted from the original 922 GeoJSON files (1.1 TB) to improve performance and accessibility. For detailed information about the conversion process and technical analysis, see [Mark Litwintschik's comprehensive blog post](https://tech.marksblogg.com/building-footprints-gba.html).

## Data Format

The dataset is organized as Parquet files, each containing building footprint data with the following key attributes:

- **Building footprints**: Vector polygons representing building outlines
- **Building heights**: Height information for each building
- **Geographic coordinates**: Precise location data
- **Metadata**: Additional building characteristics

### Parquet Structure

Each Parquet file contains:

- Building geometry (polygons)
- Height attributes
- Spatial indexing for efficient querying
- Optimized columnar storage for fast analytics

## Data Sources and Methodology

The dataset combines multiple building footprint sources:

1. **Google Open Buildings (2023)**: 1.62 billion buildings
2. **OpenStreetMap (2025)**: 490 million buildings
3. **Microsoft Building Footprints (2024)**: 432 million buildings
4. **TUM Deep Learning (ours2)**: 135 million buildings from Planet Labs satellite imagery
5. **3D Global Footprints**: 68 million buildings from previous research

The TUM deep learning component uses open source computer vision models applied to Planet Labs satellite imagery. Planet Labs operates several constellations totaling hundreds of satellites in low Earth orbit, capturing images of the entire Earth's landmasses daily.

## Data Limitations

- **Temporal Accuracy**: No timestamps for AI-generated footprints
- **Data Conflicts**: Some footprints may conflict with newer OpenStreetMap data
- **Construction Areas**: Recent construction may not be captured
- **Validation**: Manual verification recommended for critical applications

## Usage Recommendations

- **Map Making**: Use as base layer with selective integration of newer OSM data
- **Construction Areas**: Prioritize latest OSM data for areas with recent development
- **Research**: Suitable for large-scale analysis and modeling
- **Applications**: Urban planning, disaster response, infrastructure analysis

## Technical Specifications

- **Coordinate System**: WGS84
- **File Format**: Parquet
- **Compression**: Optimized for fast querying
- **Spatial Indexing**: H3 and other spatial indices included
- **Total Size**: 210 GB (compressed from 1.1 TB original)

## Data Quality Notes

The dataset combines multiple data sources as detailed above. The TUM deep learning component (ours2) represents new building footprints generated from Planet Labs satellite imagery, while the other sources provide existing building data from various organizations.

For areas with significant recent construction, consider using the latest OpenStreetMap data or Overture's building dataset, which prioritizes recent OSM updates.

## Citation

If you use this Parquet version of the dataset hosted on Source Cooperative, please cite both the original dataset and this hosted version:

**Original Dataset:**

```
@misc{1782307,
  author = {Zhu, Xiao Xiang and Chen, Sining and Zhang, Fahong and Shi, Yilei and Wang, Yuanyuan},
  title = {{GlobalBuildingAtlas: An Open Global and Complete Dataset of Building Polygons, Heights and LoD1 3D Models}},
  type = {Dataset},
  year = {2025},
  type = {Forschungsdaten},
  abstract = {GlobalBuildingAtlas is a dataset providing global and complete coverage of building polygons (GBA.Polygon), heights (GBA.Height) and Level of Detail 1 (LoD1) 3D building models (GBA.LoD1). It is the first open dataset to offer high quality, consistent, and complete building data in 2D and 3D at the individual building level on a global scale. The dataset is delivered in tiles of 5 degree by 5 degree, with GBA.Polygon and GBA.LoD1 in GeoJSON format, and GBA.Height in GeoTiff format. Details see https://github.com/zhu-xlab/GlobalBuildingAtlas.},
  keywords = {Building Footprint; Building Height; 3D Building Models; Remote Sensing; Earth Observation; Deep Learning},
  doi = {10.14459/2025mp1782307}
}
```

**Source Cooperative Hosted Version:**

```
@misc{globalbuildingatlas_lod1_parquet,
  title = {{GlobalBuildingAtlas LoD1 (Parquet Format)}},
  author = {Zhu, Xiao Xiang and Chen, Sining and Zhang, Fahong and Shi, Yilei and Wang, Yuanyuan and {Source Cooperative}},
  year = {2025},
  url = {https://source.coop/tge-labs/globalbuildingatlas-lod1},
  note = {Parquet format conversion of GlobalBuildingAtlas LoD1 dataset, hosted by Source Cooperative}
}
```


Meta CHM v2

Cloud-native version of the Version 2 High Resolution Canopy Height Maps by WRI and Meta from https://registry.opendata.aws/dataforgood-fb-forestsv2/