#postgis #duck-db #geospatial #data-transformation #load-file #process-file

bin+lib duckdb-postgis

A library for transforming geospatial data using DuckDB and ingesting it into a PostGIS database

8 releases

new 0.1.9 Jan 24, 2025
0.1.8 Jan 24, 2025
0.1.6 Dec 16, 2024
0.1.5 Oct 22, 2024

#452 in Database interfaces

Download history 409/week @ 2024-10-07 86/week @ 2024-10-14 159/week @ 2024-10-21 5/week @ 2024-11-04 12/week @ 2024-11-11 1/week @ 2024-11-18 36/week @ 2024-12-02 2/week @ 2024-12-09 170/week @ 2024-12-16 15/week @ 2024-12-23 8/week @ 2024-12-30 132/week @ 2025-01-06 42/week @ 2025-01-13 208/week @ 2025-01-20

392 downloads per month

MIT license

3MB
453 lines

DuckDB Rust lib for writing geospatial data to Postgis

Crates.io

cargo add duckdb-postgis

Current v0.1.9 release notes

This Rust library does the following things

  • Reads in a path for a geospatial data file (Geopackage, Shapefile, etc)
  • Reads in a path for non geospatial data (xlsx, xsv, parquet, etc)
  • Reads in a table name for the PostGIS database
  • Loads this file into DuckDB
  • Determines the schema and current CRS of the data - returns both
  • Performs CRS transformation on the data if required - ensures the CRS is ESPG:4326
  • Loads the data into a PostGIS table with a correctly defined geometry column

Improvements for release 0.1.10

  • Handle raster data file formats
  • Discard rows where there may be errors in the geometry column / ensure the programme doesn't crash when a geometry error is encountered - skip over it and log it instead
  • There are still bugs for loading parquet files and handling some types of geometry columns due to how they are named - these will be fixed in the next release

Example usage

mod duckdb_load;
use duckdb_load::launch_process_file;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    launch_process_file(
        "test_files/hotosm_twn_populated_places_points_geojson.geojson",
        "test-table",
        "postgresql://admin:password@localhost:5432/gridwalk",
        "test-schema",
    )?;
    Ok(())
}

Dependencies

~37MB
~459K SLoC